TREK: an integrated system architecture for intraoperative cone-beam CT-guided surgery

  • A. Uneri
  • S. Schafer
  • D. J. Mirota
  • S. Nithiananthan
  • Y. Otake
  • R. H. Taylor
  • J. H. Siewerdsen
Original Article



A system architecture has been developed for integration of intraoperative 3D imaging [viz., mobile C-arm cone-beam CT (CBCT)] with surgical navigation (e.g., trackers, endoscopy, and preoperative image and planning data). The goal of this paper is to describe the architecture and its handling of a broad variety of data sources in modular tool development for streamlined use of CBCT guidance in application-specific surgical scenarios.


The architecture builds on two proven open-source software packages, namely the cisst package (Johns Hopkins University, Baltimore, MD) and 3D Slicer (Brigham and Women’s Hospital, Boston, MA), and combines data sources common to image-guided procedures with intraoperative 3D imaging. Integration at the software component level is achieved through language bindings to a scripting language (Python) and an object-oriented approach to abstract and simplify the use of devices with varying characteristics. The platform aims to minimize offline data processing and to expose quantitative tools that analyze and communicate factors of geometric precision online. Modular tools are defined to accomplish specific surgical tasks, demonstrated in three clinical scenarios (temporal bone, skull base, and spine surgery) that involve a progressively increased level of complexity in toolset requirements.


The resulting architecture (referred to as “TREK”) hosts a collection of modules developed according to application-specific surgical tasks, emphasizing streamlined integration with intraoperative CBCT. These include multi-modality image display; 3D-3D rigid and deformable registration to bring preoperative image and planning data to the most up-to-date CBCT; 3D-2D registration of planning and image data to real-time fluoroscopy; infrared, electromagnetic, and video-based trackers used individually or in hybrid arrangements; augmented overlay of image and planning data in endoscopic or in-room video; and real-time “virtual fluoroscopy” computed from GPU-accelerated digitally reconstructed radiographs (DRRs). Application in three preclinical scenarios (temporal bone, skull base, and spine surgery) demonstrates the utility of the modular, task-specific approach in progressively complex tasks.


The design and development of a system architecture for image-guided surgery has been reported, demonstrating enhanced utilization of intraoperative CBCT in surgical applications with vastly different requirements. The system integrates C-arm CBCT with a broad variety of data sources in a modular fashion that streamlines the interface to application-specific tools, accommodates distinct workflow scenarios, and accelerates testing and translation of novel toolsets to clinical use. The modular architecture was shown to adapt to and satisfy the requirements of distinct surgical scenarios from a common code-base, leveraging software components arising from over a decade of effort within the imaging and computer-assisted interventions community.


Image-guided surgery Cone-beam CT Intraoperative imaging Surgical navigation System architecture Open-source software 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Siewerdsen JH, Moseley DJ, Burch S, Bisland SK, Bogaards A, Wilson BC, Jaffray DA (2005) Volume CT with a flat-panel detector on a mobile, isocentric C-arm: pre-clinical investigation in guidance of minimally invasive surgery. Med Phys 32(1): 241–254PubMedCrossRefGoogle Scholar
  2. 2.
    Chan Y, Siewerdsen JH, Rafferty MA, Moseley DJ, Jaffray DA, Irish JC (2008) Cone-beam computed tomography on a mobile C-arm: novel intraoperative imaging technology for guidance of head and neck surgery. J Otolaryngol Head Neck Surg 37(1): 81–90PubMedGoogle Scholar
  3. 3.
    Hamming NM, Daly MJ, Irish JC, Siewerdsen JH (2009) Automatic image-to-world registration based on X-ray projections in cone-beam CT-guided interventions. Med Phys 36(5): 1800–1812PubMedCrossRefGoogle Scholar
  4. 4.
    Zhang J, Weir V, Fajardo L, Lin J, Hsiung H, Ritenour ER (2009) Dosimetric characterization of a cone-beam O-arm imaging system. J Xray Sci Technol 17(4): 305–317PubMedGoogle Scholar
  5. 5.
    Lunsford LD, Parrish R, Albright L (1984) Intraoperative imaging with a therapeutic computed tomographic scanner. Neurosurgery 15(4): 559–561PubMedCrossRefGoogle Scholar
  6. 6.
    Jolesz FA (1998) Interventional and intraoperative MRI: a general overview of the field. J Magn Reson Imaging 8(1): 3–7PubMedCrossRefGoogle Scholar
  7. 7.
    Gary K, Ibanez L, Aylward S, Gobbi D, Blake MB, Cleary K (2006) IGSTK: an open source software toolkit for image-guided surgery. Computer 39(4): 46–53CrossRefGoogle Scholar
  8. 8.
    Deguet A, Kumar R, Taylor R, Kazanzides P (2008) The cisst libraries for computer assisted intervention systems. MIDAS J Syst Archit Comput Assist IntervGoogle Scholar
  9. 9.
    Pieper S, Lorensen B, Schroeder W, Kikinis R (2006) The NA-MIC Kit: ITK, VTK, pipelines, grids and 3D slicer as an open platform for the medical image computing community. In: Proceedings of IEEE International Symposium Biomedical Imaging, 6–9 April 2006. pp 698–701Google Scholar
  10. 10.
    Wolf I, Nolden M, Boettger T, Wegner I, Schoebinger M, Hastenteufel M, Heimann T, Meinzer H-P, Vetter M (2005) The MITK approach. Insight Journal—MICCAI Open-Source WorkshopGoogle Scholar
  11. 11.
    Rexilius J, Spindler W, Jomier J, Koenig M, Hahn H, Link F, Peitgen H-O (2005) A framework for algorithm evaluation and clinical application prototyping using ITK. Insight Journal—MICCAI Open-Source WorkshopGoogle Scholar
  12. 12.
    Schroeder W, Lorenson B (1996) Visualization toolkit: an object-oriented approach to 3-D graphics. Prentice Hall PTR, Englewood CliffsGoogle Scholar
  13. 13.
    Ibanez L, Schroeder W, Ng L, Cates J (2005) The ITK software guide. MIDAS J Syst Archit Comput Assist IntervGoogle Scholar
  14. 14.
    Tokuda J, Fischer GS, Papademetris X, Yaniv Z, Ibanez L, Cheng P, Liu H, Blevins J, Arata J, Golby AJ, Kapur T, Pieper S, Burdette EC, Fichtinger G, Tempany CM, Hata N (2009) OpenIGTLink: an open network protocol for image-guided therapy environment. Int J Med Robot 5(4): 423–434PubMedCrossRefGoogle Scholar
  15. 15.
    Rafferty MA, Siewerdsen JH, Chan Y, Daly MJ, Moseley DJ, Jaffray DA, Irish JC (2006) Intraoperative cone-beam CT for guidance of temporal bone surgery. Otolaryngol Head Neck Surg 134(5): 801–808PubMedCrossRefGoogle Scholar
  16. 16.
    Daly MJ, Siewerdsen JH, Moseley DJ, Jaffray DA, Irish JC (2006) Intraoperative cone-beam CT for guidance of head and neck surgery: assessment of dose and image quality using a C-arm prototype. Med Phys 33(10): 3767–3780PubMedCrossRefGoogle Scholar
  17. 17.
    Schafer S, Nithananiathan S, Mirota DJ, Uneri A, Stayman JW, Zbijewski W, Schmidgunst C, Kleinszig G, Khanna AJ, Siewerdsen JH (2011) Mobile C-Arm cone-beam CT for guidance of spine surgery: image quality, radiation dose, and integration with interventional guidance. Med Phys (to appear)Google Scholar
  18. 18.
    Ritter D, Orman J, Schmidgunst C, Graumann R (2007) 3D Soft tissue imaging with a mobile C-arm. Comput Med Imaging Graph 31(2): 91–102PubMedCrossRefGoogle Scholar
  19. 19.
    Nithiananthan S, Brock KK, Daly MJ, Chan H, Irish JC, Siewerdsen JH (2009) Demons deformable registration for CBCT-guided procedures in the head and neck: convergence and accuracy. Med Phys 36(10): 4755–4764PubMedCrossRefGoogle Scholar
  20. 20.
    Munbodh R, Jaffray DA, Moseley DJ, Chen Z, Knisely JPS, Cathier P, Duncan JS (2006) Automated 2D-3D registration of a radiograph and a cone beam CT using line-segment enhancement. Med Phys 33(5): 1398–1411PubMedCrossRefGoogle Scholar
  21. 21.
    Yushkevich PA, Piven J, Hazlett HC, Smith RG, Ho S, Gee JC, Gerig G (2006) User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. NeuroImage 31(3): 1116–1128PubMedCrossRefGoogle Scholar
  22. 22.
    Sherouse GW, Novins K, Chaney EL (1990) Computation of digitally reconstructed radiographs for use in radiotherapy treatment design. Int J Radiat Oncol Biol Phys 18(3): 651–658PubMedCrossRefGoogle Scholar
  23. 23.
    Reaungamornrat S, Otake Y, Uneri A, Schafer S, Stayman JW, Zbijewski W, Mirota DJ, Yoo J, Nithiananthan S, Khanna AJ, Taylor RH, Siewerdsen JH (2011) Tracker-on-C: a novel tracker configuration for image-guided therapy using a mobile C-arm. In: Computer Assisted Radiology and Surgery, Berlin, Germany, 22–25 June 2011. CARS (to appear)Google Scholar
  24. 24.
    Bradski G, Kaehler A (2008) Learning OpenCV: computer vision with the OpenCV library. O’Reilly Media, SebastopolGoogle Scholar
  25. 25.
    Oguz I, Gerig G, Barre S, Styner M (2006) KWMeshVisu: a mesh visualization tool for shape analysis. Insight Journal—MICCAI Open-Source WorkshopGoogle Scholar
  26. 26.
    Beazley DM (2003) Automated scientific software scripting with SWIG. Futur Gener Comput Syst 19(5): 599–609CrossRefGoogle Scholar
  27. 27.
    Ousterhout JK (1998) Scripting: higher level programming for the 21st Century. Computer 31(3): 23–30CrossRefGoogle Scholar
  28. 28.
    Mirota DJ, Uneri A, Schafer S, Nithiananthan S, Reh DD, Gallia GL, Taylor RH, Hager GD, Siewerdsen JH (2011) High-accuracy 3D image-based registration of endoscopic video to C-arm cone-beam CT for image-guided skull base surgery. In: Wong KH, Holmes Iii DR (eds) SPIE medical imaging: visualization, image-guided procedures, and modeling. SPIE, Lake Buena Vista, pp 79640J–79610Google Scholar
  29. 29.
    Horn BKP (1987) Closed-form solution of absolute orientation using unit quaternions. J Opt Soc Am A 4(4): 629–642CrossRefGoogle Scholar
  30. 30.
    Park FC, Martin BJ (1994) Robot sensor calibration: solving AX=XB on the Euclidean group. IEEE Trans Robot Autom 10(5): 717–721CrossRefGoogle Scholar
  31. 31.
    Daly MJ, Chan H, Prisman E, Vescan A, Nithiananthan S, Qiu J, Weersink R, Irish JC, Siewerdsen JH (2010) Fusion of intraoperative cone-beam CT and endoscopic video for image-guided procedures. In: Wong KH, Miga MI (eds) SPIE medical imaging: visualization, image-guided procedures, and modeling. SPIE, San Diego, pp 762503–762508Google Scholar
  32. 32.
    Mirota D, Wang H, Taylor R, Ishii M, Hager G (2009) Toward video-based navigation for endoscopic endonasal skull base surgery. In: Yang G-Z, Hawkes D, Rueckert D, Noble A, Taylor C (eds) Medical Image Computing and Computer Assisted Intervention Lecture notes in computer science, vol 5761. Springer, Berlin, Heidelberg, pp 91–99Google Scholar
  33. 33.
    Strobl KH, Hirzinger G (2006) Optimal hand-eye calibration. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems, 9–15 Oct. 2006, pp 4647–4653Google Scholar
  34. 34.
    Siddon RL (1985) Fast calculation of the exact radiological path for a three-dimensional CT array. Med Phys 12(2): 252–255PubMedCrossRefGoogle Scholar
  35. 35.
    Pluim J, Maintz J, Viergever M (2000) Image registration by maximization of combined mutual information and gradient information. In: Delp S, DiGoia A, Jaramaz B (eds) Medical Image Computing and Computer Assisted Intervention Lecture notes in computer science, vol 1935. Springer, Berlin, Heidelberg, pp 103–129Google Scholar
  36. 36.
    Hansen N (2006) The CMA evolution strategy: a comparing review. In: Lozano J, Larrañaga P, Inza I, Bengoetxea E (eds) Towards a new evolutionary computation, vol 192. Studies in fuzziness and soft computing. Springer, Berlin, Heidelberg, pp 75–102Google Scholar
  37. 37.
    Fitzpatrick JM, West JB, Maurer CR Jr (1998) Predicting error in rigid-body point-based registration. IEEE Trans Med Imaging 17(5): 694–702PubMedCrossRefGoogle Scholar
  38. 38.
    Simpson AL, Ma B, Ellis RE, Stewart AJ, Miga MI (2011) Uncertainty propagation and analysis of image-guided surgery. In: Wong KH, Holmes Iii DR (eds) SPIE medical imaging: visualization, image-guided procedures, and modeling. SPIE, Lake Buena Vista, Florida, USA, pp 79640H–79647Google Scholar
  39. 39.
    Hamming NM, Daly MJ, Irish JC, Siewerdsen JH (2008) Effect of fiducial configuration on target registration error in intraoperative cone-beam CT guidance of head and neck surgery. In: Proceedings of the IEEE Engineering in Medicine and Biology Society, 20–25 Aug. 2008, pp 3643–3648Google Scholar
  40. 40.
    Higgins WE, Helferty JP, Lu K, Merritt SA, Rai L, Yu K-C (2008) 3D CT-video fusion for image-guided bronchoscopy. Comput Med Imaging Graph 32(3): 159–173PubMedCrossRefGoogle Scholar
  41. 41.
    DiGioia AM 3rd, Jaramaz B, Colgan BD (1998) Computer assisted orthopaedic surgery. Image guided and robotic assistive technologies. Clin Orthop Relat Res 354: 8–16PubMedCrossRefGoogle Scholar

Copyright information

© CARS 2011

Authors and Affiliations

  • A. Uneri
    • 1
  • S. Schafer
    • 2
  • D. J. Mirota
    • 3
  • S. Nithiananthan
    • 2
  • Y. Otake
    • 4
  • R. H. Taylor
    • 5
  • J. H. Siewerdsen
    • 6
    • 7
  1. 1.Department of Computer ScienceJohns Hopkins UniversityBaltimoreUSA
  2. 2.Department of Biomedical EngineeringJohns Hopkins UniversityBaltimoreUSA
  3. 3.Department of Computer ScienceJohns Hopkins UniversityNE BaltimoreUSA
  4. 4.Department of Computer ScienceJohns Hopkins UniversityNE BaltimoreUSA
  5. 5.Department of Computer ScienceJohns Hopkins UniversityNE BaltimoreUSA
  6. 6.Department of Computer ScienceJohns Hopkins UniversityBaltimoreUSA
  7. 7.Department of Biomedical EngineeringJohns Hopkins UniversityBaltimoreUSA

Personalised recommendations