The Visual Computer

, Volume 29, Issue 6–8, pp 805–815 | Cite as

Visibility-driven PET-CT visualisation with region of interest (ROI) segmentation

  • Younhyun Jung
  • Jinman KimEmail author
  • Stefan Eberl
  • Micheal Fulham
  • David Dagan Feng
Original Article


Multi-modality (MM) positron emission tomography-computed tomography (PET-CT) visualises biological and physiological functions (from PET) as region of interests (ROIs) within a higher resolution anatomical reference frame (from CT). The need to efficiently assess and assimilate the information from these co-aligned volumes simultaneously has stimulated new visualisation techniques that combine 3D volume rendering with interactive transfer functions to enable efficient manipulation of these volumes. However, in typical MM volume rendering visualisation, the transfer functions for the volumes are manipulated in isolation with the resulting volumes being fused, thus failing to exploit the spatial correlation that exists between the aligned volumes. Such lack of feedback makes MM transfer function manipulation complex and time consuming. Further, transfer function alone is often insufficient to select the ROIs when they have similar voxel properties to those of non-relevant regions.

In this study, we propose a new ROI-based MM visibility-driven transfer function (m 2-vtf) for PET-CT visualisation. We present a novel ‘visibility’ metric, a fundamental optical property that represents how much of the ROIs are visible to the users, and use it to measure the visibility of the ROIs in PET in relation to how it is affected by transfer function manipulations to its counterpart CT. To overcome the difficulty in ROI selection, we provide an intuitive ROI selection tool based on automated PET segmentation. We further present a MM transfer function automation where the visibility metrics from the PET ROIs are used to automate its CT’s transfer function. Our GPU implementation achieved an interactive visualisation of PET-CT with efficient and intuitive transfer function manipulations.


Multi-modality volume rendering Visibility histogram Transfer function PET-CT imaging Image segmentation 



We would like to thank our collaborators at the Royal Prince Alfred (RPA) Hospital. This research was funded by Australian Research Council (ARC) grants.


  1. 1.
    Rosset, A., Spadola, L., Pysher, L., Ratib, O.: Navigating the fifth dimension: innovative interface for multidimensional multimodality image navigation. Radiograph 26(1), 299–308 (2006) CrossRefGoogle Scholar
  2. 2.
    Preim, B., Bartz, D.: Visualization in Medicine Theory, Algorithms, and Application. Morgan Kaufmann Series in Computer Graphics. Morgan Kaufmann, San Mateo (2007) Google Scholar
  3. 3.
    Kindlmann, G., Durkin, J.W.: Semi-automatic generation of transfer functions for direct volume rendering. In: Proc. IEEE Visualization, pp. 79–86 (1998) Google Scholar
  4. 4.
    Kindlmann, G., Whitaker, R., Tasdizen, T., Moller, T.: Curvature-based transfer functions for direct volume rendering: methods and applications. In: Proc. IEEE Visualization, pp. 513–520 (2003) Google Scholar
  5. 5.
    Kniss, J., Kindlmann, G., Hansen, C.: Interactive volume rendering using multi-dimensional transfer functions and direct manipulation widgets. In: Proc. IEEE Visualization, pp. 255–262 (2001) Google Scholar
  6. 6.
    Correa, C.D., Ma, K.: The occlusion spectrum for volume classification and visualization. IEEE Trans. Vis. Comput. Graph. 15(6), 1465–1472 (2009) CrossRefGoogle Scholar
  7. 7.
    Caban, J.J., Rheingans, P.: Texture-based transfer functions for direct volume rendering. IEEE Trans. Vis. Comput. Graph. 14(6), 1364–1371 (2008) CrossRefGoogle Scholar
  8. 8.
    Correa, C.D., Ma, K.: Size-based transfer functions: a new volume exploration technique. IEEE Trans. Vis. Comput. Graph. 14(6), 1380–1387 (2008) CrossRefGoogle Scholar
  9. 9.
    Cai, W., Sakas, W.: Data intermixing and multi-volume rendering. Comput. Graph. Forum 18(3), 359–368 (1999) CrossRefGoogle Scholar
  10. 10.
    Bramon, R., Bardera, A., Rodriguez, J., Feixas, M., Puig, J., Sbert, M.: Multimodal data fusion based on mutual information. IEEE Trans. Vis. Comput. Graph. 18(9), 1574–1587 (2012) CrossRefGoogle Scholar
  11. 11.
    Haidacher, M., Bruckner, S., Kanitsar, A., Grller, M.E.: Information-based transfer functions for multimodal visualization. In: Proc. Visual Comput. Biomed, pp. 101–108 (2008) Google Scholar
  12. 12.
    Kim, J., Eberl, S., Feng, D.: Visualizing dual-modality rendered volumes using a dual-lookup table transfer function. Comput. Sci. Eng. 9(1), 20–25 (2007) CrossRefGoogle Scholar
  13. 13.
    Correa, C.D., Ma, K.: Visibility histograms and visibility-driven transfer functions. IEEE Trans. Vis. Comput. Graph. 17(2), 192–204 (2011) CrossRefGoogle Scholar
  14. 14.
    Ruiz, M., Bardera, A., Boada, I., Viola, I., Feixas, M., Sbert, M.: Automatic transfer functions based on informational divergence. IEEE Trans. Vis. Comput. Graph. 17(12), 1932–1941 (2011) CrossRefGoogle Scholar
  15. 15.
    Wang, Y., Zhang, J., Chen, W., Zhang, H., Chi, X.: Efficient opacity specification based on feature visibilities in direct volume rendering. Comput. Graph. Forum 30(7), 2117–2126 (2011) CrossRefGoogle Scholar
  16. 16.
    Jung, Y., Kim, J., Feng, D.: Dual-modal visibility metrics for interactive PET-CT visualization. Proc. IEEE EMBC, 2696–2699 (2012) Google Scholar
  17. 17.
    Bordoloi, U., Shen, H.W.: View selection for volume rendering. In: Proc. IEEE Visualization, pp. 487–494 (2005) Google Scholar
  18. 18.
    Viola, I., Kanitsar, A., Eduard, M.: Importance-driven feature enhancement in volume visualization. IEEE Trans. Vis. Comput. Graph. 11(4), 408–418 (2005) CrossRefGoogle Scholar
  19. 19.
    Tzeng, F.Y., Ma, K.: A cluster-space visual interface for arbitrary dimensional classification of volume data. In: Eurographics. IEEE TCVG Symposium on Visualization, pp. 17–24 (2004) Google Scholar
  20. 20.
    Kim, J., Cai, W., Eberl, S., Feng, D.: Real-time volume rendering visualization of dual-modality PET/CT images with interactive fuzzy thresholding segmentation. IEEE Trans. Inf. Technol. Biomed. 11(2), 161–169 (2007) CrossRefGoogle Scholar
  21. 21.
    Wahl, R., Jacene, H., Kasamon, Y., Lodge, M.: From RECIST to PERCIST: evolving considerations for PET response criteria in solid tumors. J. Nucl. Med. 50(Suppl. 1), 122S–150S (2009) CrossRefGoogle Scholar
  22. 22.
    Bi, L., Kim, J., Wen, L., Feng, D.: Automated and robust PERCIST-based thresholding framework for whole body PET-CT studies. Proc. IEEE EMBC, 5335–5338 (2012) Google Scholar
  23. 23.
    Nelder, J.A., Mead, R.: A simplex method for function minimization. Comput. J. 7, 303–313 (1965) MathSciNetCrossRefGoogle Scholar
  24. 24.
    Barton, R.R., Ivey, J.S.: Nelder-mead simplex modifications for simulation optimization. INFORMS J. Comput. 42(7), 954–973 (1996) zbMATHGoogle Scholar
  25. 25.
    Lagarias, J.C., Reeds, J.A.: Convergence properties of the Nelder–Mead simplex method in low dimensions. SIAM J. Optim. 9(1), 112–147 (1998) MathSciNetzbMATHCrossRefGoogle Scholar
  26. 26.
    Manousopoulos, P., Michalopoulos, M.: Comparison of non-linear optimization algorithms for yield curve estimation. Eur. J. Oper. Res. 192(2), 594–602 (2009) MathSciNetzbMATHCrossRefGoogle Scholar
  27. 27.
    Voreen: volume rendering engine.
  28. 28.
    Maintz, J.B.A., Viegever, M.A.: A survey of medical image registration. Med. Image Anal. 2(1), 1–36 (1998) CrossRefGoogle Scholar
  29. 29.
    Kim, J., Feng, D.D., Cai, T.W., Eberl, S.: Automatic 3D temporal kinetics segmentation of dynamic emission tomography image using adaptive region growing cluster analysis. In: IEEE Proc. NSS-MIC, pp. 1580–1583 (2002) Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Younhyun Jung
    • 1
  • Jinman Kim
    • 1
    Email author
  • Stefan Eberl
    • 1
    • 2
  • Micheal Fulham
    • 1
    • 2
    • 3
  • David Dagan Feng
    • 1
    • 4
  1. 1.Biomedical and Multimedia Information Technology (BMIT) Research GroupUniversity of SydneySydneyAustralia
  2. 2.Department of Molecular ImagingRoyal Prince Alfred HospitalSydneyAustralia
  3. 3.Sydney Medical SchoolUniversity of SydneySydneyAustralia
  4. 4.Med-X Research InstituteShanghai Jiao Tong UniversityShanghaiChina

Personalised recommendations