Skip to main content
SpringerLink
Log in

Bilinear Models for Spatio-Temporal Point Distribution Analysis

Application to Extrapolation of Left Ventricular, Biventricular and Whole Heart Cardiac Dynamics

  • Published:
International Journal of Computer Vision Aims and scope Submit manuscript

Abstract

In this work we describe the usage of bilinear statistical models as a means of factoring the shape variability into two components attributed to inter-subject variation and to the intrinsic dynamics of the human heart. We show that it is feasible to reconstruct the shape of the heart at discrete points in the cardiac cycle. Provided we are given a small number of shape instances representing the same heart at different points in the same cycle, we can use the bilinear model to establish this.

Using a temporal and a spatial alignment step in the preprocessing of the shapes, around half of the reconstruction errors were on the order of the axial image resolution of 2 mm, and over 90% was within 3.5 mm. From this, we conclude that the dynamics were indeed separated from the inter-subject variability in our dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abboud, B., & Davoine, F. (2004). Bilinear factorization for facial expression analysis and synthesis. IEE Proceedings—Vision, Image and Signal Processing, 152(3), 327–333.

    Article  Google Scholar 

  • Bistoquet, A., Oshinski, J., & Škrinjar, O. (2007). Left ventricular deformation recovery from cine MRI using an incompressible model. IEEE Transactions on Medical Imaging, 26(9), 1136–1153.

    Article  Google Scholar 

  • Blackall, J. M., King, A. P., Penney, G. P., Adam, A., & Hawkes, D. J. (2001). A statistical model of respiratory motion and deformation of the liver. In W. J. Niessen & M. A. Viergever (Eds.), Lecture notes in computer science: Vol. 2208. Proc. 4th int. conf. medical image computing and computer assisted intervention (MICCAI), Utrecht, The Netherlands (pp. 1338–1340). Berlin: Springer.

    Google Scholar 

  • Bosch, J. G., Mitchell, S. C., Lelieveldt, B. P. F., Nijland, F., Kamp, O., Sonka, M., & Reiber, J. H. C. (2002). Automatic segmentation of echocardiographic sequences by active appearance motion models. IEEE Transactions on Medical Imaging, 21(11), 1374–1383.

    Article  Google Scholar 

  • Chandrashekara, R., Rao, A., Sanchez-Ortiz, G. I., Mohiaddin, R. H., & Rueckert, D. (2003). Construction of a statistical model for cardiac motion analysis using nonrigid image registration. In C. J. Taylor & J. A. Noble (Eds.), Lecture notes in computer science: Vol. 2732. Proc. 18th int. conf. information processing in medical imaging (IPMI), Ambleside, United Kingdom (pp. 599–610). Berlin: Springer.

    Google Scholar 

  • Chuang, E., & Bregler, C. (2005). Mood swings: Expressive speech animation. ACM Transactions on Graphics, 24(2), 331–347.

    Article  Google Scholar 

  • Cootes, T. F., Cooper, D. H., Taylor, C. J., & Graham, J. (1992). Trainable method of parametric shape description. Image & Vision Computing, 10(5), 289–294.

    Article  Google Scholar 

  • Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(6), 681–685.

    Article  Google Scholar 

  • Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995). Active shape models—their training and application. Computer Vision and Image Understanding, 61(1), 38–59.

    Article  Google Scholar 

  • Cuzzolin, F. (2006). Using bilinear models for view-invariant action and identity recognition. In Proc. IEEE int. conf. on computer vision and pattern recognition (CVPR), New York, NY, USA (pp. 1701–1708) 2006.

  • De Lathauwer, L., De Moor, B., & Vandewalle, J. (2000). A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4), 1253–1278.

    Article  MATH  MathSciNet  Google Scholar 

  • Dryden, I. L., & Mardia, K. V. (1998). Statistical shape analysis. New York: Wiley.

    MATH  Google Scholar 

  • Duncan, J. S., & Ayache, N. (2000). Medical image analysis: progress over two decades and the challenges ahead. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1), 85–106.

    Article  Google Scholar 

  • Frangi, A. F., Niessen, W. J., Viergever, M. A., & Lelieveldt, B. P. F. (2005). A survey of three-dimensional modeling techniques for quantitative functional analysis of cardiac images. In L. Landini, V. Positano, & M.F. Santarelli (Eds.), Advanced image processing in magnetic resonance imaging (Chap. 9, pp. 267–342). Boca Raton: CRC Press.

    Google Scholar 

  • González-Mora, J., De la Torre, F., Murthi, R., Guil, N., & Zapata, E. L. (2007). Bilinear active appearance models. In R. Goecke, S. Lucey, & I. Matthews (Eds.), Proc. int. workshop on non-rigid registration and tracking through learning, Rio de Janeiro, Brazil, 2007.

  • Goodall, C. (1991). Procrustes methods in shape analysis. Journal of the Royal Statistical Society—Series B: Statistical Methodology, 53(2), 285–339.

    MATH  MathSciNet  Google Scholar 

  • Grimes, D. B., & Rao, R. P. N. (2005). Bilinear sparse coding for invariant vision. Neural Computation, 17(1), 47–73.

    Article  Google Scholar 

  • Hamarneh, G., & Gustavsson, T. (2004). Deformable spatio-temporal shape models: extending active shape models to 2D + time. Image & Vision Computing, 22(6), 461–470.

    Article  Google Scholar 

  • Hoogendoorn, C., Sukno, F. M., Ordás, S., & Frangi, A. F. (2007). Bilinear models for spatio-temporal point distribution analysis: application to extrapolation of whole heart cardiac dynamics. In: M. Nielsen, W. Niessen & C. F. Westin (Eds.), Proc. 8th int. workshop on mathematical methods in biomedical image analysis (MMBIA), Rio de Janeiro, Brazil, 2007.

  • Hsu, E., Pulli, K., & Popović, J. (2005). Style translation for human motion. ACM Transactions on Graphics, 23(3), 1082–1089.

    Article  Google Scholar 

  • Kendall, D. G. (1984). Shape manifolds, procrustean metrics and complex projective spaces. Bulletin of the London Mathematical Society, 16(2), 81–121.

    Article  MATH  MathSciNet  Google Scholar 

  • Le, H., & Kendall, D. G. (1993). The Riemannian structure of euclidean shape spaces: A novel environment for statistics. Annals of Statistics, 21(3), 1225–1271.

    Article  MATH  MathSciNet  Google Scholar 

  • Lee, C. S., & Elgammal, A. (2004). Gait style and gait content: Bilinear models for gait recognition using gait resampling. In Proc. 6th IEEE int. conf. on automatic face and gesture recognition (FGR), Seoul, Korea (pp. 147–152) 2007.

  • Lekadir, K., Keenan, N., Pennell, D., & Yang, G. Z. (2007). Shape-based myocardial contractility analysis using multivariate outlier detection. In N. Ayache, S. Ourselin, & A. Maeder (Eds.), Lecture notes in computer science: Vol. 4792. Proc. 10th int. conf. medical image computing and computer assisted intervention (MICCAI), Brisbane, QLD, Australia (pp. 834–841). Berlin: Springer.

    Google Scholar 

  • Leung, K. Y. E., & Bosch, J. G. (2007). Localized shape variations for classifying wall motion in echocardiograms. In N. Ayache, S. Ourselin, & A. Maeder (Eds.), Lecture notes in computer science: Vol. 4791. Proc. 10th int. conf. medical image computing and computer assisted intervention (MICCAI), Brisbane, QLD, Australia (pp. 52–59). Berlin: Springer.

    Google Scholar 

  • Liu, H., & Shi, P. (2007). State-space analysis of cardiac motion with biomechanical constraints. IEEE Transactions on Image Processing, 16(4), 901–916.

    Article  MathSciNet  Google Scholar 

  • Lynch, M., Ghita, O., & Whelan, P. (2008). Segmentation of the left ventricle of the heart in 3D + t MRI data using an optimised non-rigid temporal model. IEEE Transactions on Medical Imaging, 27(2), 195–203.

    Article  Google Scholar 

  • Magnus, J. R., & Neudecker, H. (1988). Matrix differential calculus with applications in statistics and econometrics. New York: Wiley.

    MATH  Google Scholar 

  • Mardia, K. V., & Dryden, I. L. (1989). Shape distributions for landmark data. Advances in Applied Probability, 21(4), 742–755.

    Article  MATH  MathSciNet  Google Scholar 

  • Marimont, D. H., & Wandell, B. A. (1992). Linear models of surface and illuminant spectra. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 9(11), 1905–1913.

    Article  Google Scholar 

  • McInerney, T., & Terzopoulos, D. (1996). Deformable models in medical image analysis: a survey. Medical Image Analysis, 1(2), 91–108.

    Article  Google Scholar 

  • Mitchell, S. C., Bosch, J. G., Lelieveldt, B. P. F., van der Geest, R. J., Reiber, J. H. C., & Sonka, M. (2002). 3-D active appearance models: segmentation of cardiac MR and ultrasound images. IEEE Transactions on Medical Imaging, 21(9), 1167–1178.

    Article  Google Scholar 

  • Montagnat, J., & Delingette, H. (2005). 4D deformable models with temporal constraints: application to 4D cardiac image segmentation. Medical Image Analysis, 9(1), 87–100.

    Article  Google Scholar 

  • Ohnesorge, B. M., Becker, C. R., Flohr, T. G., & Reiser, M. F. (2002). Multi-slice CT in cardiac imaging: technical principles, clinical application and future developments. Berlin: Springer.

    Google Scholar 

  • Ordás, S., Oubel, E., Leta, R., Carrera, F., & Frangi, A. F. (2007). A statistical shape model of the heart and its application to model-based segmentation. In A. Manduca & X. P. Hu (Eds.), Proc. SPIE medical imaging, San Diego, CA, USA, Vol. 6511.

  • Pantic, M., & Rothkrantz, L. J. M. (2000). Automatic analysis of facial expressions: the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), 1424–1445.

    Article  Google Scholar 

  • Perperidis, D. (2005). Spatio-temporal registration and modelling of the heart using cardiovascular MR imaging. Ph.D. thesis, Imperial College London.

  • Perperidis, D., Mohiaddin, R. H., & Rueckert, D. (2005). Spatio-temporal free-form registration of cardiac MR image sequences. Medical Image Analysis, 9(5), 441–456.

    Article  Google Scholar 

  • Shin, D., Lee, H. S., & Kim, D. (2008). Illumination-robust face recognition using ridge regressive bilinear models. Pattern Recognition Letters, 29(1), 49–58.

    Article  Google Scholar 

  • Styner, M. A., Rajamani, K. T., Nolte, L. P., Zsemliye, G., Székely, G., Taylor, C. J., & Davies, R. H. (2003). Evaluation of 3D correspondence methods for model building. In Lecture notes in computer science: Vol. 2732. Proc. 18th int. conf. information processing in medical imaging (IPMI), Ambleside, United Kingdom (pp. 63–75). Berlin: Springer.

    Google Scholar 

  • Suri, J. S. (2000). Computer vision, pattern recognition and image processing in left ventricle segmentation: the last 50 years. Pattern Analysis & Applications, 3(3), 209–242.

    Article  MATH  Google Scholar 

  • Syeda-Mahmood, T., Wang, F., Beymer, D., London, M., & Reddy, R. (2007). Characterizing spatio-temporal patterns for disease discrimination in cardiac echo videos. In N. Ayache, S. Ourselin, A. Maeder (Eds.), Lecture notes in computer science: Vol. 4791. Proc. 10th int. conf. medical image computing and computer assisted intervention (MICCAI), Brisbane, QLD, Australia (pp. 261–269). Berlin: Springer.

    Google Scholar 

  • Tenenbaum, J. B., & Freeman, W. T. (1996). Separating style and content. In M. Mozer, M. I. Jordan & T. Petsche (Eds.), Advances in neural information processing systems, Denver, CO, USA (pp. 662–668) 1996.

  • Tenenbaum, J. B., & Freeman, W. T. (2000). Separating style and content with bilinear models. Neural Computation, 12(6), 1247–1283.

    Article  Google Scholar 

  • Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 71–86.

    Article  Google Scholar 

  • Vasilescu, M. A. O., & Terzopoulos, D. (2003). Multilinear subspace analysis of image ensembles. In Proc. IEEE int. conf. on computer vision and pattern recognition (CVPR), Madison, WI, USA (pp. II: 93–99), 2006.

Download references

Author information

Authors and Affiliations

  1. Networking Biomedical Research Center on Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN), Barcelona, Spain

    Corné Hoogendoorn, Federico M. Sukno & Alejandro F. Frangi

  2. Center for Computational Imaging and Simulation Technologies in Biomedicine (CISTIB), Department of Information and Communication Technologies, Universitat Pompeu Fabra, Carrer Roc Boronat 138, 08018, Barcelona, Spain

    Corné Hoogendoorn, Federico M. Sukno, Sebastián Ordás & Alejandro F. Frangi

Authors
  1. Corné Hoogendoorn
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Federico M. Sukno
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sebastián Ordás
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alejandro F. Frangi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alejandro F. Frangi.

Additional information

The work of A.F.F. was supported by the Spanish Ministry of Education and Science under a Ramon y Cajal Research Fellowship. This work was partially developed within the framework of the CENIT-CDTEAM Project funded by the Spanish CDTI-MITYC, and also partially supported by grants MEC TEC2006-03617/TCM and ISCIII FIS2004/40676.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hoogendoorn, C., Sukno, F.M., Ordás, S. et al. Bilinear Models for Spatio-Temporal Point Distribution Analysis. Int J Comput Vis 85, 237–252 (2009). https://doi.org/10.1007/s11263-009-0212-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11263-009-0212-6

Keywords

Search

Navigation