The Visual Computer

, Volume 31, Issue 11, pp 1431–1446 | Cite as

A parallelized 4D reconstruction algorithm for vascular structures and motions based on energy optimization

  • Xinglong Liu
  • Fei HouEmail author
  • Aimin Hao
  • Hong Qin
Original Article


In this paper, we present a parallel 4D vessel reconstruction algorithm that simultaneously recovers 3D structure, shape, and motion based on multiple views of X-ray angiograms. The fundamental goal is to assist the analysis and diagnosis of interventional surgery in the most efficient way towards interactive and accurate performance. We start with a fully parallelized algorithm to extract vessels as well as their skeletons and topologies from dynamic image sequences. Then, instead of resorting to registration, we present an algorithm to formulate the reconstruction problem as an energy minimization problem with color, coherence, and topology constraints to reconstruct the 3D vessel initially, which is robust to combat noise and incomplete information in images. Next, we incorporate temporal information into our energy optimization framework to track and reconstruct 4D kinematics of the dynamic vessels, which is also capable of recovering previous incomplete and misleading shapes acquired from static images otherwise. We demonstrate our system in coronary arteries reconstruction and movement tracking for percutaneous coronary intervention surgery to help medical practitioners learn about the 3D shapes and their motions of the coronary arteries of specific patient. We envision that our system would be of high assistance for diagnosis and therapy to treat vessel-related diseases in a clinical setting in the near future.


X-ray angiograms 3D reconstruction Motion tracking Belief propagation 



This work was supported in part by National Natural Science Foundation of China (Grant No. 61190120, 61190121, 61190125, 61300068, 61300067), National Science Foundation of USA (Grant No. IIS-0949467, IIS-1047715, and IIS-1049448), the National High Technology Research and Development Program (863 Program) of China (Grant No. 012AA011503), Postdoctoral Science Foundation of China (Grant No. 2013M530512).

Supplementary material

ESM 1 (MP4 85,218 kb)


  1. 1.
    Sarry, L., Boire, J.-Y.: Three-dimensional tracking of coronary arteries from biplane angiographic sequences using parametrically deformable models. IEEE Trans. Med. Imaging 20(12), 1341–1351 (2001)CrossRefGoogle Scholar
  2. 2.
    Cañero, C., Vilariño, F., Mauri, J., Radeva, P.: Predictive (un) distortion model and 3-d reconstruction by biplane snakes. IEEE Trans. Med. Imaging 21(9), 1188–1201 (2002)CrossRefGoogle Scholar
  3. 3.
    Dumay, A.C., Gerbrands, J.J., Reiber, J.H.: Automated extraction, labelling and analysis of the coronary vasculature from arteriograms. Int. J. Card. Imaging 10(3), 205–215 (1994)CrossRefGoogle Scholar
  4. 4.
    Aylward, S.R., Bullitt, E.: Initialization, noise, singularities, and scale in height ridge traversal for tubular object centerline extraction. IEEE Trans. Med. Imaging 21(2), 61–75 (2002)CrossRefGoogle Scholar
  5. 5.
    Sen, A., Lan, L., Doi, K., Hoffmann, K.R.: Quantitative evaluation of vessel tracking techniques on coronary angiograms. Med. Phys. 26(5), 698–706 (1999)CrossRefGoogle Scholar
  6. 6.
    Kirbas, C., Quek, F.: A review of vessel extraction techniques and algorithms. ACM Comput. Surv. (CSUR) 36(2), 81–121 (2004)CrossRefGoogle Scholar
  7. 7.
    Hoover, A., Kouznetsova, V., Goldbaum, M.: Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans. Med. Imaging 19(3), 203–210 (2000)CrossRefGoogle Scholar
  8. 8.
    Li, Q., You, J., Zhang, D.: Vessel segmentation and width estimation in retinal images using multiscale production of matched filter responses. Expert Syst. Appl. 39(9), 7600–7610 (2012)CrossRefGoogle Scholar
  9. 9.
    Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multiscale vessel enhancement filtering. In: MICCAI’98, Springer. pp. 130–137 (1998)Google Scholar
  10. 10.
    Condurache, A.-P., Aach, T.: Vessel segmentation in angiograms using hysteresis thresholding. In: IAPR Conference on Machine Vision Applications, pp. 269–272 (2005)Google Scholar
  11. 11.
    Zhang, B., Zhang, L., Zhang, L., Karray, F.: Retinal vessel extraction by matched filter with first-order derivative of gaussian. Comput. Biol. Med. 40(4), 438–445 (2010)CrossRefGoogle Scholar
  12. 12.
    Van Uitert, R., Bitter, I.: Subvoxel precise skeletons of volumetric data based on fast marching methods. Med. Phys. 34(2), 627–638 (2007)CrossRefGoogle Scholar
  13. 13.
    Hassouna, M., Farag, A.: Multistencils fast marching methods: a highly accurate solution to the eikonal equation on cartesian domains. IEEE Trans. Pattern Anal. Mach. Intell. 29(9), 1563–1574 (2007). doi: 10.1109/TPAMI.2007.1154 CrossRefGoogle Scholar
  14. 14.
    Zhang, T.Y., Suen, C.Y.: A fast parallel algorithm for thinning digital patterns. Commun. ACM 27(3), 236–239 (1984)CrossRefGoogle Scholar
  15. 15.
    Wellnhofer, E., Wahle, A., Mugaragu, I., Gross, J., Oswald, H., Fleck, E.: Validation of an accurate method for three-dimensional reconstruction and quantitative assessment of volumes, lengths and diameters of coronary vascular branches and segments from biplane angiographic projections. Int. J. Card. Imaging 15(5), 339–353 (1999)CrossRefGoogle Scholar
  16. 16.
    Messenger, J.C., Chen, S.J., Carroll, J.D., Burchenal, J., Kioussopoulos, K., Groves, B.M.: 3d coronary reconstruction from routine single-plane coronary angiograms: clinical validation and quantitative analysis of the right coronary artery in 100 patients. Int. J. Card. Imaging 16(6), 413–427 (2000)CrossRefGoogle Scholar
  17. 17.
    Gollapudi, R.R., Valencia, R., Lee, S.S., Wong, G.B., Teirstein, P.S., Price, M.J.: Utility of three-dimensional reconstruction of coronary angiography to guide percutaneous coronary intervention. Catheter. Cardiovasc. Interv. 69(4), 479–482 (2007)CrossRefGoogle Scholar
  18. 18.
    Movassaghi, B., Rasche, V., Grass, M., Viergever, M.A., Niessen, W.J.: A quantitative analysis of 3-d coronary modeling from two or more projection images. IEEE Trans. Med. Imaging 23(12), 1517–1531 (2004)CrossRefGoogle Scholar
  19. 19.
    Sprague, K., Drangova, M., Lehmann, G., Slomka, P., Levin, D., Chow, B., et al.: Coronary x-ray angiographic reconstruction and image orientation. Med. Phys. 33, 707–718 (2006)CrossRefGoogle Scholar
  20. 20.
    Hansis, E., Schäfer, D., Dössel, O., Grass, M.: Projection-based motion compensation for gated coronary artery reconstruction from rotational x-ray angiograms. Phys. Med. Biol. 53(14), 3807–3820 (2008)CrossRefGoogle Scholar
  21. 21.
    Nguyen, T.V., Sklansky, J.: Reconstructing the 3-d medial axes of coronary arteries in single-view cineangiograms. IEEE Trans. Med. Imaging 13(1), 61–73 (1994)CrossRefGoogle Scholar
  22. 22.
    Fessler, J.A., Macovski, A.: Object-based 3-d reconstruction of arterial trees from magnetic resonance angiograms. IEEE Trans. Med. Imaging 10, 25–39 (1991)CrossRefGoogle Scholar
  23. 23.
    Liu, I., Sun, Y.: Fully automated reconstruction of three-dimensional vascular tree structures from two orthogonal views using computational algorithms and productionrules. Opt. Eng. 31(10), 2197–2207 (1992)CrossRefGoogle Scholar
  24. 24.
    Weng, J., Ahuja, N., Huang, T.S.: Optimal motion and structure estimation. IEEE Trans. Pattern Anal. Mach. Intell. 15(9), 864–884 (1993)CrossRefGoogle Scholar
  25. 25.
    Chen, S.-Y.J., Hoffmann, K.R., Carroll, J.D.: Three-dimensional reconstruction of coronary arterial tree based on biplane angiograms. Proc. SPIE Med. Imag. Image Process. 2710, 103–114 (1996)Google Scholar
  26. 26.
    Chen, S.Y.J., Metz, C.E.: Improved determination of biplane imaging geometry from two projection images and its application to 3-d reconstruction of coronary arterial trees. Med. Phys. 24, 633–654 (1997)CrossRefGoogle Scholar
  27. 27.
    Ruan, S., Bruno, A., Coatrieux, J.-L.: Three-dimensional motion and reconstruction of coronary arteries from biplane cineangiography. Image Vis. Comput. 12(10), 683–689 (1994)CrossRefGoogle Scholar
  28. 28.
    Puentes, J., Roux, C., Garreau, M., Coatrieux, J.-L.: Dynamic feature extraction of coronary artery motion using dsa image sequences. IEEE Trans. Med. Imaging 17(6), 857–871 (1998)CrossRefGoogle Scholar
  29. 29.
    Ingrassia, C., Windyga, P., Shah, M.: Segmentation and tracking of coronary arteries. In: BMES/EMBS Conference, vol. 1, IEEE, pp. 203–203 (1999)Google Scholar
  30. 30.
    Chen, S.-Y., Carroll, J.D.: Kinematic and deformation analysis of 4-d coronary arterial trees reconstructed from cine angiograms. IEEE Trans. Med. Imaging 22(6), 710–721 (2003)CrossRefGoogle Scholar
  31. 31.
    Naegel, B., Passat, N., Ronse, C.: Grey-level hit-or-miss transforms-Part I: unified theory. Pattern Recognit. 40(2), 635–647 (2007)zbMATHCrossRefGoogle Scholar
  32. 32.
    Shechter, G., Devernay, F., Coste-Manière, E., Quyyumi, A., McVeigh, E.R.: Three-dimensional motion tracking of coronary arteries in biplane cineangiograms. IEEE Trans. Med. Imaging 22(4), 493–503 (2003)CrossRefGoogle Scholar
  33. 33.
    Shechter, G., Resar, J.R., McVeigh, E.R.: Displacement and velocity of the coronary arteries: cardiac and respiratory motion. IEEE Trans. Med. Imaging 25(3), 369–375 (2006)CrossRefGoogle Scholar
  34. 34.
    Blondel, C., Malandain, G., Vaillant, R., Ayache, N.: Reconstruction of coronary arteries from a single rotational x-ray projection sequence. IEEE Trans. Med. Imaging 25(5), 653–663 (2006)CrossRefGoogle Scholar
  35. 35.
    Bouattour, S., Arndt, R., Paulus, D.: 4D reconstruction of coronary arteries from monoplane angiograms. In: Computer Analysis of Images and Patterns, Springer. pp. 724–731 (2005)Google Scholar
  36. 36.
    Schoonenberg, G., Florent, R., Lelong, P., Wink, O., Ruijters, D., Carroll, J., ter Haar, B.: Projection-based motion compensation and reconstruction of coronary segments and cardiac implantable devices using rotational x-ray angiography. Med. Image Anal. 13(5), 785–792 (2009)CrossRefGoogle Scholar
  37. 37.
    Geman, S., Geman, D.: Stochastic relaxation, gibbs distributions, and the bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721–741 (1984)zbMATHCrossRefGoogle Scholar
  38. 38.
    Lafferty, J.D., McCallum, A., Pereira, F.C.N.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning, ICML ’01, Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, pp. 282–289 (2001)Google Scholar
  39. 39.
    Pearl, J.: Reverend bayes on inference engines: a distributed hierarchical approach. AAAI. pp. 133–136 (1982)Google Scholar
  40. 40.
    Meltzer, T., Yanover, C., Weiss, Y.: Globally optimal solutions for energy minimization in stereo vision using reweighted belief propagation. In: Proceedings of the Tenth IEEE International Conference on Computer Vision, ICCV ’05, IEEE Computer Society, pp. 428–435 (2005)Google Scholar
  41. 41.
    Murphy, K.P., Weiss, Y., Jordan, M.I.: Loopy belief propagation for approximate inference: An empirical study. In: Proceedings of the Fifteenth conference on Uncertainty in artificial intelligence, Morgan Kaufmann Publishers Inc. pp. 467–475 (1999)Google Scholar
  42. 42.
    Szeliski, R., Zabih, R., Scharstein, D., Veksler, O., Kolmogorov, V., Agarwala, A., Tappen, M., Rother, C.: A comparative study of energy minimization methods for markov random fields with smoothness-based priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(6), 1068–1080 (2008)Google Scholar
  43. 43.
    Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical mrf parameters. In: Ninth IEEE International Conference on Computer Vision, IEEE, pp. 900–906 (2003)Google Scholar
  44. 44.
    Potetz, B., Lee, T.S.: Efficient belief propagation for higher-order cliques using linear constraint nodes. Comput. Vis. Image Underst. 112(1), 39–54 (2008)CrossRefGoogle Scholar
  45. 45.
    Brunton, A., Shu, C., Roth, G.: Belief propagation on the gpu for stereo vision. In: The 3rd Canadian Conference on Computer and Robot Vision, pp. 76–76 (2006)Google Scholar
  46. 46.
    Coughlan, J., Shen, H.: Dynamic quantization for belief propagation in sparse spaces. Comput. Vis. Image Underst. 106(1), 47–58 (2007)CrossRefGoogle Scholar
  47. 47.
    Felzenszwalb, P.F., Huttenlocher, D.P.: Efficient belief propagation for early vision. Int. J. Comput. Vis. 70(1), 41–54 (2006)CrossRefGoogle Scholar
  48. 48.
    Boykov, Y.Y., Jolly, M.-P.: Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In: Eighth IEEE International Conference on Computer Vision, vol. 1, pp. 105–112 (2001)Google Scholar
  49. 49.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  50. 50.
    Kolmogorov, V., Zabin, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)CrossRefGoogle Scholar
  51. 51.
    Vezhnevets, V., Konouchine, V.: Growcut: Interactive multi-label nd image segmentation by cellular automata. In: Proc. of Graphicon, pp. 150–156 (2005)Google Scholar
  52. 52.
    Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Trans. Gr. (TOG) ACM. 23, 309–314 (2004)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  1. 1.State Key Laboratory of Virtual Reality Technology and SystemsBeihang UniversityBeijingChina
  2. 2.Department of Computer ScienceStony Brook UniversityStony BrookUSA

Personalised recommendations