Skip to main content
SpringerLink
Log in

Simulating Developmental Cardiac Morphology in Virtual Reality Using a Deformable Image Registration Approach

  • Published:
Annals of Biomedical Engineering Aims and scope Submit manuscript

Abstract

While virtual reality (VR) has potential in enhancing cardiovascular diagnosis and treatment, prerequisite labor-intensive image segmentation remains an obstacle for seamlessly simulating 4-dimensional (4-D, 3-D + time) imaging data in an immersive, physiological VR environment. We applied deformable image registration (DIR) in conjunction with 3-D reconstruction and VR implementation to recapitulate developmental cardiac contractile function from light-sheet fluorescence microscopy (LSFM). This method addressed inconsistencies that would arise from independent segmentations of time-dependent data, thereby enabling the creation of a VR environment that fluently simulates cardiac morphological changes. By analyzing myocardial deformation at high spatiotemporal resolution, we interfaced quantitative computations with 4-D VR. We demonstrated that our LSFM-captured images, followed by DIR, yielded average dice similarity coefficients of 0.92 ± 0.05 (n = 510) and 0.93 ± 0.06 (n = 240) when compared to ground truth images obtained from Otsu thresholding and manual segmentation, respectively. The resulting VR environment simulates a wide-angle zoomed-in view of motion in live embryonic zebrafish hearts, in which the cardiac chambers are undergoing structural deformation throughout the cardiac cycle. Thus, this technique allows for an interactive micro-scale VR visualization of developmental cardiac morphology to enable high resolution simulation for both basic and clinical science.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5

Similar content being viewed by others

References

  1. Abiri, A., A. Tao, M. LaRocca, X. Guan, S. Askari, J. Bisley, E. Dutson, and W. Grundfest. Visual–perceptual mismatch in robotic surgery. Surg. Endosc. 31:3271–3278, 2016.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Araki, T., N. Ikeda, N. Dey, S. Chakraborty, L. Saba, D. Kumar, E. Godia, X. Jiang, A. Gupta, P. Radeva, J. Laird, A. Nicolaides, and J. Suri. A comparative approach of four different image registration techniques for quantitative assessment of coronary artery calcium lesions using intravascular ultrasound. Comput. Methods Programs Biomed. 118:158–172, 2015.

    Article  PubMed  Google Scholar 

  3. Brock, K. K. Results of a multi-institution deformable registration accuracy study (MIDRAS). Int. J. Radiat. Oncol. Biol. Phys. 76:583–596, 2010.

    Article  PubMed  Google Scholar 

  4. Brock, K. K., L. A. Dawson, M. B. Sharpe, D. J. Moseley, and D. A. Jaffray. Feasibility of a novel deformable image registration technique to facilitate classification, targeting, and monitoring of tumor and normal tissue. Int. J. Radiat. Oncol. Biol. Phys. 64:1245–1254, 2006.

    Article  PubMed  Google Scholar 

  5. Brock, K., M. Sharpe, L. Dawson, S. Kim, and D. Jaffray. Accuracy of finite element model-based multi-organ deformable image registration. Med. Phys. 32:1647–1659, 2005.

    Article  CAS  PubMed  Google Scholar 

  6. Bronstein, A. M., M. M. Bronstein, R. Kimmel, M. Mahmoudi, and G. Sapiro. A Gromov-Hausdorff framework with diffusion geometry for topologically-robust non-rigid shape matching. Int. J. Comp. Vis. 89:266–286, 2010.

    Article  Google Scholar 

  7. Brown, L. A survey of image registration techniques. ACM Comput. Surv. 24:325–376, 1992.

    Article  Google Scholar 

  8. Chan, S., F. Conti, K. Salisbury, and N. Blevins. Virtual reality simulation in neurosurgery: technologies and evolution. Neurosurgery 72:A154–A164, 2013.

    Article  Google Scholar 

  9. Cuchet, E., J. Knoplioch, D. Dormont, and C. Marsault. Registration in neurosurgery and neuroradiotherapy applications. J. Image Guid. Surg. 1:198–207, 1995.

    Article  CAS  PubMed  Google Scholar 

  10. Dallal, G. An analytic approximation to the distribution of Lilliefors’s test statistic for normality. Am. Stat. 40:294–296, 1986.

    Google Scholar 

  11. Ding, Y., A. Abiri, P. Abiri, S. Li, C.-C. Chang, K. I. Baek, J. J. Hsu, E. Sideris, Y. Li, J. Lee, T. Segura, T. P. Nguyen, A. Bui, and R. R. Sevag. Integrating light-sheet imaging with virtual reality to recapitulate developmental cardiac mechanics. JCI Insight 2:22, 2017.

    Article  Google Scholar 

  12. Ding, Y., J. Lee, J. J. Hsu, C. C. Chang, K. I. Baek, S. Ranjbarvaziri, R. Ardehali, R. R. S. Packard, and T. K. Hsiai. Light-sheet imaging to elucidate cardiovascular injury and repair. Curr. Cardiol. Rep. 20:35, 2018.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Ding, Y., H. Xie, T. Peng, Y. Lu, D. Jin, J. Teng, Q. Ren, and P. Xi. Laser oblique scanning optical microscopy (LOSOM) for phase relief imaging. Opt. Express 20:14100–14108, 2012.

    Article  PubMed  Google Scholar 

  14. Ding, Y., M. Zhang, J. Lang, J. Leng, Q. Ren, J. Yang, and C. Li. In vivo study of endometriosis in mice by photoacoustic microscopy. J. Biophotonics 8:94–101, 2015.

    Article  PubMed  Google Scholar 

  15. Fei, P., J. Lee, R. Sevag Packard, K.-I. Sereti, H. Xu, J. Ma, Y. Ding, H. Kang, H. Chen, K. Sung, R. Kulkarni, R. Ardehali, J. Kuo, X. Xu, C.-M. Ho, and T. Hsiai. Cardiac light-sheet fluorescent microscopy for multi-scale and rapid imaging of architecture and function. Sci. Rep. 6:22489, 2016.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. Fei, P., J. Nie, J. Lee, Y. Ding, S. Li, Z. Yu, H. Zhang, M. Hagiwara, T. Yu, T. Segura, C.-M. Ho, D. Zhu, and T. K. Hsiai. Sub-voxel light-sheet microscopy for high-resolution, high-throughput volumetric imaging of large biomedical specimens. bioRxiv, 2018. http://biorxiv.org/content/early/2018/01/29/255695.abstract

  17. Freeborough, P. A., R. P. Woods, and N. C. Fox. Accurate registration of serial 3D MR brain images and its application to visualizing change in neurodegenerative disorders. J. Comput. Assist. Tomogr. 20:1012–1022, 1996.

    Article  CAS  PubMed  Google Scholar 

  18. Fuchs, E., J. Jaffe, R. Long, and F. Azam. Thin laser light sheet microscope for microbial oceanography. Opt. Express 10:145–154, 2002.

    Article  PubMed  Google Scholar 

  19. Gallagher, A. G., and C. U. Cates. Virtual reality training for the operating room and cardiac catheterisation laboratory. Lancet 364:1538–1540, 2004.

    Article  PubMed  Google Scholar 

  20. Greenbaum, P. The lawnmower man. Film Video 9:58–62, 1992.

    Google Scholar 

  21. Gu, X., H. Pan, Y. Liang, R. Castillo, D. Yang, D. Choi, E. Castillo, A. Majumdar, T. Guerrero, and S. Jiang. Implementation and evaluation of various demons deformable image registration algorithms on a GPU. Phys. Med. Biol. 55:207–219, 2010.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Guiraudon, G. M., D. L. Jones, D. Bainbridge, and T. M. Peters. Mitral valve implantation using off-pump closed beating intracardiac surgery: a feasibility study. Interact. Cardiovasc. Thorac. Surg. 6:603–607, 2007.

    Article  PubMed  Google Scholar 

  23. Handels, H., and J. Ehrhardt. Medical image computing for computer-supported diagnostics and therapy. Methods Inf. Med. 48:11–17, 2009.

    Article  CAS  PubMed  Google Scholar 

  24. Hill, D., P. Batchelor, M. Holden, and D. Hawkes. Medical image registration. Phys. Med. Biol. 46:R1, 2001.

    Article  CAS  PubMed  Google Scholar 

  25. Holden, M., D. Hill, E. Denton, J. Jarosz, T. Cox, T. Rohlfing, J. Goodey, and D. Hawkes. Voxel similarity measures for 3-D serial MR brain image registration. IEEE Trans. Med. Imaging 19:94–102, 2000.

    Article  CAS  PubMed  Google Scholar 

  26. Huisken, J., J. Swoger, F. Del Bene, J. Wittbrodt, and E. H. K. Stelzer. Optical sectioning deep inside live embryos by selective plane illumination microscopy. Science. 305:1007–1009, 2004.

    Article  CAS  PubMed  Google Scholar 

  27. Hwang, S. S., H.-D. Kim, T. Y. Jang, J. Yoo, S. Kim, K. Paeng, and S. D. Kim. Image-based object reconstruction using run-length representation. Signal Proc. Image Commun. 51:1–12, 2017.

    Article  Google Scholar 

  28. Kanade, T., and P. J. Narayanan. Virtualized reality: perspectives on 4D digitization of dynamic events. IEEE Comp. Graph. Appl. 27:32–40, 2007.

    Article  Google Scholar 

  29. Kardell, M., M. Magnusson, M. Sandborg, G. Alm Carlsson, J. Jeuthe, and A. Malusek. Automatic segmentation of pelvis for brachytherapy of prostate. Radiat. Prot. Dosimetr. 169:398–404, 2016.

    Article  CAS  Google Scholar 

  30. Keller, P. J., A. D. Schmidt, J. Wittbrodt, and E. H. K. Stelzer. Reconstruction of zebrafish early embryonic development by scanned light sheet microscopy. Science. 322:1065–1069, 2008.

    Article  CAS  PubMed  Google Scholar 

  31. King, F., J. Jayender, S. Bhagavatula, P. Shyn, S. Pieper, T. Kapur, A. Lasso, and G. Fichtinger. An Immersive Virtual Reality Environment for Diagnostic Imaging. J. Med. Robot. Res. 1:1640003-1–9, 2016.

    Article  Google Scholar 

  32. Lee, J., P. Fei, R. Sevag Packard, H. Kang, H. Xu, K. I. Baek, N. Jen, J. Chen, H. Yen, J. Kuo, N. Chi, C.-M. Ho, and T. Hsiai. 4-Dimensional light-sheet microscopy to elucidate shear stress modulation of cardiac trabeculation. J. Clin. Invest. 126:1679–1690, 2016.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Lemole, G., P. Banerjee, C. Luciano, S. Neckrysh, and F. Charbel. Virtual reality in neurosurgical education. Neurosurgery 61:142–149, 2007.

    Article  PubMed  Google Scholar 

  34. Li, G., D. Citrin, K. Camphausen, B. Mueller, C. Burman, B. Mychalczak, R. W. Miller, and Y. Song. Advances in 4D medical imaging and 4D radiation therapy. Technol. Cancer Res. Treat. 7:67–81, 2008.

    Article  CAS  PubMed  Google Scholar 

  35. Li, Z., M. Wu, W. Zhou, and J. Yu. 4D human body correspondences from panoramic depth maps. 2018.

  36. Lipman, Y., and T. Funkhouser. Möbius voting for surface correspondence. ACM T. Graph. 28:72, 2009.

    Article  Google Scholar 

  37. Litman, R., and A. M. Bronstein. Learning spectral descriptors for deformable shape correspondence. IEEE T. Patt. Anal. Mach. Intell. 36:171–180, 2014.

    Article  CAS  Google Scholar 

  38. Lorenzo-Valdés, M., G. I. Sanchez-Ortiz, R. Mohiaddin, and D. Rueckert. Atlas-based segmentation and tracking of 3D cardiac MR images using non-rigid registration BT—medical image computing and computer-assisted intervention—MICCAI 2002. Berlin: Springer, 2002.

    Google Scholar 

  39. Lu, W., P. J. Parikh, I. M. El Naqa, M. M. Nystrom, J. P. Hubenschmidt, S. H. Wahab, S. Mutic, A. K. Singh, G. E. Christensen, J. D. Bradley, and D. A. Low. Quantitation of the reconstruction quality of a four-dimensional computed tomography process for lung cancer patients. Med. Phys. 32:890–901, 2005.

    Article  PubMed  Google Scholar 

  40. Lu, Y., K. Yang, K. Zhou, B. Pang, G. Wang, Y. Ding, Q. Zhang, H. Han, J. Tian, C. Li, and Q. Ren. An integrated Quad-modality molecular imaging system for small animals. J. Nucl. Med. 55:1375–1379, 2014.

    Article  CAS  PubMed  Google Scholar 

  41. Mcinerney, T., and D. Terzopoulos. A dynamic finite element surface model for segmentation and tracking in multidimensional medical images with application to cardiac 4D image analysis. Comput. Med. Imaging Graph. 19:69–83, 1995.

    Article  CAS  PubMed  Google Scholar 

  42. Metz, C. T., S. Klein, M. Schaap, T. van Walsum, and W. J. Niessen. Nonrigid registration of dynamic medical imaging data using nD + t B-splines and a groupwise optimization approach. Med. Image Anal. 15:238–249, 2011.

    Article  CAS  PubMed  Google Scholar 

  43. Mitchell, S. C., J. G. Bosch, B. P. F. Lelieveldt, R. J. Van der Geest, J. H. C. Reiber, and M. Sonka. 3-D active appearance models: segmentation of cardiac MR and ultrasound images. IEEE T. Med. Imaging 21:1167–1178, 2002.

    Article  Google Scholar 

  44. Montagnat, J., and H. Delingette. 4D deformable models with temporal constraints: application to 4D cardiac image segmentation. Med. Image Anal. 9:87–100, 2005.

    Article  PubMed  Google Scholar 

  45. Otsu, N. A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man. Cybern. 9:62–66, 1979.

    Article  Google Scholar 

  46. Packard, R. R. S., K. I. Baek, T. Beebe, N. Jen, Y. DIng, F. Shi, P. Fei, B. J. Kang, P. H. Chen, J. Gau, M. Chen, J. Y. Tang, Y. H. Shih, Y. DIng, D. Li, X. Xu, and T. K. Hsiai. Automated segmentation of light-sheet fluorescent imaging to characterize experimental doxorubicin-induced cardiac injury and repair. Sci. Rep. 7:1–11, 2017.

    Article  CAS  Google Scholar 

  47. Peng, H., Z. Ruan, F. Long, J. H. Simpson, and E. W. Myers. V3D enables real-time 3D visualization and quantitative analysis of large-scale biological image data sets. Nat. Biotechnol. 28:348–353, 2010.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  48. Peng, H., J. Tang, H. Xiao, A. Bria, J. Zhou, V. Butler, Z. Zhou, P. T. Gonzalez-Bellido, S. W. Oh, J. Chen, A. Mitra, R. W. Tsien, H. Zeng, G. A. Ascoli, G. Iannello, M. Hawrylycz, E. Myers, and F. Long. Virtual finger boosts three-dimensional imaging and microsurgery as well as terabyte volume image visualization and analysis. Nat. Commun. 5:1–13, 2014.

    Google Scholar 

  49. Planchon, T., L. Gao, D. Milkie, M. Davidson, J. Galbraith, C. Galbraith, and E. Betzig. Rapid three-dimensional isotropic imaging of living cells using Bessel beam plane illumination. Nat. Methods 8:417–423, 2011.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  50. Pottmann, H., J. Wallner, Q.-X. Huang, and Y.-L. Yang. Integral invariants for robust geometry processing. Comp. Aided Geom. Des. 26:37–60, 2009.

    Article  Google Scholar 

  51. Power, R. M., and J. Huisken. A guide to light-sheet fluorescence microscopy for multiscale imaging. Nat. Meth. 14:360–373, 2017.

    Article  CAS  Google Scholar 

  52. Reznick, R., and H. MacRae. Teaching surgical skills—changes in the wind. N. Engl. J. Med. 355:2664–2669, 2006.

    Article  CAS  PubMed  Google Scholar 

  53. Riva, G. Applications of virtual environments in medicine. Methods Inf. Med. 42:524–534, 2003.

    Article  CAS  PubMed  Google Scholar 

  54. Shen, J.-K., B. Matuszewski, L.-K. Shark, A. Skalski, T. Zielinski, and C. Moore. Deformable image registration—a critical evaluation: demons, B-Spline FFD and spring mass system. 2008.

  55. Smith, L. N., A. R. Farooq, M. L. Smith, I. E. Ivanov, and A. Orlando. Realistic and interactive high-resolution 4D environments for real-time surgeon and patient interaction. Int. J. Med. Robot. 13:e1761, 2017.

    Article  Google Scholar 

  56. Thirion, J. Image matching as a diffusion process: an analogy with Maxwell’s demons. Med. Image Anal. 2:243–260, 1998.

    Article  CAS  PubMed  Google Scholar 

  57. Turinsky, A. L., and C. W. Sensen. On the way to building an integrated computational environment for the study of developmental patterns and genetic diseases. Int. J. Nanomed. 1:89, 2006.

    Article  Google Scholar 

  58. van der Meijden, O., and M. Schijven. The value of haptic feedback in conventional and robot-assisted minimal invasive surgery and virtual reality training: a current review. Surg. Endosc. 23:1180–1190, 2009.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Vemuri, A. S., J. C.-H. Wu, K.-C. Liu, and H.-S. Wu. Deformable three-dimensional model architecture for interactive augmented reality in minimally invasive surgery. Surg. Endosc. 26:3655–3662, 2012.

    Article  PubMed  Google Scholar 

  60. Verveer, P., J. Swoger, F. Pampaloni, K. Greger, M. Marcello, and E. Stelzer. High-resolution three-dimensional imaging of large specimens with light sheet-based microscopy. Nat. Methods 4:311–313, 2007.

    Article  CAS  PubMed  Google Scholar 

  61. Weissleder, R., and M. Pittet. Imaging in the era of molecular oncology. Nature 452:580–589, 2008.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  62. Wierzbicki, M., M. Drangova, G. Guiraudon, and T. Peters. Validation of dynamic heart models obtained using non-linear registration for virtual reality training, planning, and guidance of minimally invasive cardiac surgeries. Med. Image Anal. 8:387–401, 2004.

    Article  PubMed  Google Scholar 

  63. Yan, D. Adaptive radiotherapy: merging principle into clinical practice. Semin. Radiat. Oncol. 20:79–83, 2010.

    Article  PubMed  Google Scholar 

  64. Yan, D., F. Vicini, J. Wong, and A. Martinez. Adaptive radiation therapy. Phys. Med. Biol. 43:123, 1997.

    Article  Google Scholar 

  65. Yang, J. C., C. H. Chen, and M. C. Jeng. Integrating video-capture virtual reality technology into a physically interactive learning environment for English learning. Comput. Educ. 55:1346–1356, 2010.

    Article  Google Scholar 

  66. Zou, K. H., S. K. Warfield, A. Bharatha, C. M. C. Tempany, M. R. Kaus, S. J. Haker, W. M. W. Iii, and F. A. Jolesz. Statistical validation of image segmentation quality based on a spatial overlap index. Sci. Rep. 11:178–189, 2006.

    Google Scholar 

Download references

Acknowledgments

This work was supported by the National Institutes of Health (5R01HL083015-10, 1R01HL118650, 1R01HL129727, 7R01HL111437) and the American Heart Association (Career Development Award 18CDA34110338, Scientist Development Grant 16SDG30910007).

Conflict of interest

The authors declare no competing financial interests.

Author information

Author notes

  1. Arash Abiri and Yichen Ding have contributed equally to this work.

Authors and Affiliations

  1. Department of Bioengineering, University of California, Los Angeles, CA, 90095, USA

    Arash Abiri, Yichen Ding, Parinaz Abiri & Tzung K. Hsiai

  2. Department of Medicine, David Geffen School of Medicine at UCLA, Los Angeles, CA, 90095, USA

    Arash Abiri, Yichen Ding, Parinaz Abiri, René R. Sevag Packard & Tzung K. Hsiai

  3. Medical Engineering, California Institute of Technology, Pasadena, CA, 91125, USA

    Tzung K. Hsiai

  4. Department of Biomedical Engineering, University of California, Irvine, CA, 92697, USA

    Arash Abiri

  5. Department of Pediatrics (Cardiology), Stanford University, Stanford, CA, 94305, USA

    Vijay Vedula & Alison Marsden

  6. Department of Bioengineering, Stanford University, Stanford, CA, 94305, USA

    Alison Marsden

  7. Institute for Computational and Mathematical Engineering, Stanford University, Stanford, CA, 94305, USA

    Alison Marsden

  8. Department of Electrical Engineering, University of Southern California, Los Angeles, CA, 90089, USA

    C.-C. Jay Kuo

Authors
  1. Arash Abiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yichen Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Parinaz Abiri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. René R. Sevag Packard
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Vijay Vedula
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alison Marsden
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. C.-C. Jay Kuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Tzung K. Hsiai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tzung K. Hsiai.

Additional information

Associate Editor Konstantinos Konstantopoulos oversaw the review of this article.

Electronic supplementary material

Below is the link to the electronic supplementary material.

10439_2018_2113_MOESM1_ESM.mov

4-D VR Simulation of contracting embryonic zebrafish heart. Video of the 4-D VR simulation of a contracting embryonic zebrafish heart throughout an entire cardiac cycle. The VR scene was generated and visualized using the Unity engine. Supplementary material 1 (MOV 7180 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Abiri, A., Ding, Y., Abiri, P. et al. Simulating Developmental Cardiac Morphology in Virtual Reality Using a Deformable Image Registration Approach. Ann Biomed Eng 46, 2177–2188 (2018). https://doi.org/10.1007/s10439-018-02113-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10439-018-02113-z

Keywords

Search

Navigation