International Journal of Computer Vision

, Volume 85, Issue 3, pp 237–252 | Cite as

Bilinear Models for Spatio-Temporal Point Distribution Analysis

Application to Extrapolation of Left Ventricular, Biventricular and Whole Heart Cardiac Dynamics
  • Corné Hoogendoorn
  • Federico M. Sukno
  • Sebastián Ordás
  • Alejandro F. FrangiEmail author


In this work we describe the usage of bilinear statistical models as a means of factoring the shape variability into two components attributed to inter-subject variation and to the intrinsic dynamics of the human heart. We show that it is feasible to reconstruct the shape of the heart at discrete points in the cardiac cycle. Provided we are given a small number of shape instances representing the same heart at different points in the same cycle, we can use the bilinear model to establish this.

Using a temporal and a spatial alignment step in the preprocessing of the shapes, around half of the reconstruction errors were on the order of the axial image resolution of 2 mm, and over 90% was within 3.5 mm. From this, we conclude that the dynamics were indeed separated from the inter-subject variability in our dataset.


Statistical shape modeling Cardiac modeling Cardiac dynamics Bilinear models Spatiotemporal decomposition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abboud, B., & Davoine, F. (2004). Bilinear factorization for facial expression analysis and synthesis. IEE Proceedings—Vision, Image and Signal Processing, 152(3), 327–333. CrossRefGoogle Scholar
  2. Bistoquet, A., Oshinski, J., & Škrinjar, O. (2007). Left ventricular deformation recovery from cine MRI using an incompressible model. IEEE Transactions on Medical Imaging, 26(9), 1136–1153. CrossRefGoogle Scholar
  3. Blackall, J. M., King, A. P., Penney, G. P., Adam, A., & Hawkes, D. J. (2001). A statistical model of respiratory motion and deformation of the liver. In W. J. Niessen & M. A. Viergever (Eds.), Lecture notes in computer science: Vol. 2208. Proc. 4th int. conf. medical image computing and computer assisted intervention (MICCAI), Utrecht, The Netherlands (pp. 1338–1340). Berlin: Springer. Google Scholar
  4. Bosch, J. G., Mitchell, S. C., Lelieveldt, B. P. F., Nijland, F., Kamp, O., Sonka, M., & Reiber, J. H. C. (2002). Automatic segmentation of echocardiographic sequences by active appearance motion models. IEEE Transactions on Medical Imaging, 21(11), 1374–1383. CrossRefGoogle Scholar
  5. Chandrashekara, R., Rao, A., Sanchez-Ortiz, G. I., Mohiaddin, R. H., & Rueckert, D. (2003). Construction of a statistical model for cardiac motion analysis using nonrigid image registration. In C. J. Taylor & J. A. Noble (Eds.), Lecture notes in computer science: Vol. 2732. Proc. 18th int. conf. information processing in medical imaging (IPMI), Ambleside, United Kingdom (pp. 599–610). Berlin: Springer. Google Scholar
  6. Chuang, E., & Bregler, C. (2005). Mood swings: Expressive speech animation. ACM Transactions on Graphics, 24(2), 331–347. CrossRefGoogle Scholar
  7. Cootes, T. F., Cooper, D. H., Taylor, C. J., & Graham, J. (1992). Trainable method of parametric shape description. Image & Vision Computing, 10(5), 289–294. CrossRefGoogle Scholar
  8. Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(6), 681–685. CrossRefGoogle Scholar
  9. Cootes, T. F., Taylor, C. J., Cooper, D. H., & Graham, J. (1995). Active shape models—their training and application. Computer Vision and Image Understanding, 61(1), 38–59. CrossRefGoogle Scholar
  10. Cuzzolin, F. (2006). Using bilinear models for view-invariant action and identity recognition. In Proc. IEEE int. conf. on computer vision and pattern recognition (CVPR), New York, NY, USA (pp. 1701–1708) 2006. Google Scholar
  11. De Lathauwer, L., De Moor, B., & Vandewalle, J. (2000). A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4), 1253–1278. zbMATHCrossRefMathSciNetGoogle Scholar
  12. Dryden, I. L., & Mardia, K. V. (1998). Statistical shape analysis. New York: Wiley. zbMATHGoogle Scholar
  13. Duncan, J. S., & Ayache, N. (2000). Medical image analysis: progress over two decades and the challenges ahead. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(1), 85–106. CrossRefGoogle Scholar
  14. Frangi, A. F., Niessen, W. J., Viergever, M. A., & Lelieveldt, B. P. F. (2005). A survey of three-dimensional modeling techniques for quantitative functional analysis of cardiac images. In L. Landini, V. Positano, & M.F. Santarelli (Eds.), Advanced image processing in magnetic resonance imaging (Chap. 9, pp. 267–342). Boca Raton: CRC Press. Google Scholar
  15. González-Mora, J., De la Torre, F., Murthi, R., Guil, N., & Zapata, E. L. (2007). Bilinear active appearance models. In R. Goecke, S. Lucey, & I. Matthews (Eds.), Proc. int. workshop on non-rigid registration and tracking through learning, Rio de Janeiro, Brazil, 2007. Google Scholar
  16. Goodall, C. (1991). Procrustes methods in shape analysis. Journal of the Royal Statistical Society—Series B: Statistical Methodology, 53(2), 285–339. zbMATHMathSciNetGoogle Scholar
  17. Grimes, D. B., & Rao, R. P. N. (2005). Bilinear sparse coding for invariant vision. Neural Computation, 17(1), 47–73. CrossRefGoogle Scholar
  18. Hamarneh, G., & Gustavsson, T. (2004). Deformable spatio-temporal shape models: extending active shape models to 2D + time. Image & Vision Computing, 22(6), 461–470. CrossRefGoogle Scholar
  19. Hoogendoorn, C., Sukno, F. M., Ordás, S., & Frangi, A. F. (2007). Bilinear models for spatio-temporal point distribution analysis: application to extrapolation of whole heart cardiac dynamics. In: M. Nielsen, W. Niessen & C. F. Westin (Eds.), Proc. 8th int. workshop on mathematical methods in biomedical image analysis (MMBIA), Rio de Janeiro, Brazil, 2007. Google Scholar
  20. Hsu, E., Pulli, K., & Popović, J. (2005). Style translation for human motion. ACM Transactions on Graphics, 23(3), 1082–1089. CrossRefGoogle Scholar
  21. Kendall, D. G. (1984). Shape manifolds, procrustean metrics and complex projective spaces. Bulletin of the London Mathematical Society, 16(2), 81–121. zbMATHCrossRefMathSciNetGoogle Scholar
  22. Le, H., & Kendall, D. G. (1993). The Riemannian structure of euclidean shape spaces: A novel environment for statistics. Annals of Statistics, 21(3), 1225–1271. zbMATHCrossRefMathSciNetGoogle Scholar
  23. Lee, C. S., & Elgammal, A. (2004). Gait style and gait content: Bilinear models for gait recognition using gait resampling. In Proc. 6th IEEE int. conf. on automatic face and gesture recognition (FGR), Seoul, Korea (pp. 147–152) 2007. Google Scholar
  24. Lekadir, K., Keenan, N., Pennell, D., & Yang, G. Z. (2007). Shape-based myocardial contractility analysis using multivariate outlier detection. In N. Ayache, S. Ourselin, & A. Maeder (Eds.), Lecture notes in computer science: Vol. 4792. Proc. 10th int. conf. medical image computing and computer assisted intervention (MICCAI), Brisbane, QLD, Australia (pp. 834–841). Berlin: Springer. Google Scholar
  25. Leung, K. Y. E., & Bosch, J. G. (2007). Localized shape variations for classifying wall motion in echocardiograms. In N. Ayache, S. Ourselin, & A. Maeder (Eds.), Lecture notes in computer science: Vol. 4791. Proc. 10th int. conf. medical image computing and computer assisted intervention (MICCAI), Brisbane, QLD, Australia (pp. 52–59). Berlin: Springer. Google Scholar
  26. Liu, H., & Shi, P. (2007). State-space analysis of cardiac motion with biomechanical constraints. IEEE Transactions on Image Processing, 16(4), 901–916. CrossRefMathSciNetGoogle Scholar
  27. Lynch, M., Ghita, O., & Whelan, P. (2008). Segmentation of the left ventricle of the heart in 3D + t MRI data using an optimised non-rigid temporal model. IEEE Transactions on Medical Imaging, 27(2), 195–203. CrossRefGoogle Scholar
  28. Magnus, J. R., & Neudecker, H. (1988). Matrix differential calculus with applications in statistics and econometrics. New York: Wiley. zbMATHGoogle Scholar
  29. Mardia, K. V., & Dryden, I. L. (1989). Shape distributions for landmark data. Advances in Applied Probability, 21(4), 742–755. zbMATHCrossRefMathSciNetGoogle Scholar
  30. Marimont, D. H., & Wandell, B. A. (1992). Linear models of surface and illuminant spectra. Journal of the Optical Society of America A: Optics, Image Science, and Vision, 9(11), 1905–1913. CrossRefGoogle Scholar
  31. McInerney, T., & Terzopoulos, D. (1996). Deformable models in medical image analysis: a survey. Medical Image Analysis, 1(2), 91–108. CrossRefGoogle Scholar
  32. Mitchell, S. C., Bosch, J. G., Lelieveldt, B. P. F., van der Geest, R. J., Reiber, J. H. C., & Sonka, M. (2002). 3-D active appearance models: segmentation of cardiac MR and ultrasound images. IEEE Transactions on Medical Imaging, 21(9), 1167–1178. CrossRefGoogle Scholar
  33. Montagnat, J., & Delingette, H. (2005). 4D deformable models with temporal constraints: application to 4D cardiac image segmentation. Medical Image Analysis, 9(1), 87–100. CrossRefGoogle Scholar
  34. Ohnesorge, B. M., Becker, C. R., Flohr, T. G., & Reiser, M. F. (2002). Multi-slice CT in cardiac imaging: technical principles, clinical application and future developments. Berlin: Springer. Google Scholar
  35. Ordás, S., Oubel, E., Leta, R., Carrera, F., & Frangi, A. F. (2007). A statistical shape model of the heart and its application to model-based segmentation. In A. Manduca & X. P. Hu (Eds.), Proc. SPIE medical imaging, San Diego, CA, USA, Vol. 6511. Google Scholar
  36. Pantic, M., & Rothkrantz, L. J. M. (2000). Automatic analysis of facial expressions: the state of the art. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(12), 1424–1445. CrossRefGoogle Scholar
  37. Perperidis, D. (2005). Spatio-temporal registration and modelling of the heart using cardiovascular MR imaging. Ph.D. thesis, Imperial College London. Google Scholar
  38. Perperidis, D., Mohiaddin, R. H., & Rueckert, D. (2005). Spatio-temporal free-form registration of cardiac MR image sequences. Medical Image Analysis, 9(5), 441–456. CrossRefGoogle Scholar
  39. Shin, D., Lee, H. S., & Kim, D. (2008). Illumination-robust face recognition using ridge regressive bilinear models. Pattern Recognition Letters, 29(1), 49–58. CrossRefGoogle Scholar
  40. Styner, M. A., Rajamani, K. T., Nolte, L. P., Zsemliye, G., Székely, G., Taylor, C. J., & Davies, R. H. (2003). Evaluation of 3D correspondence methods for model building. In Lecture notes in computer science: Vol. 2732. Proc. 18th int. conf. information processing in medical imaging (IPMI), Ambleside, United Kingdom (pp. 63–75). Berlin: Springer. Google Scholar
  41. Suri, J. S. (2000). Computer vision, pattern recognition and image processing in left ventricle segmentation: the last 50 years. Pattern Analysis & Applications, 3(3), 209–242. zbMATHCrossRefGoogle Scholar
  42. Syeda-Mahmood, T., Wang, F., Beymer, D., London, M., & Reddy, R. (2007). Characterizing spatio-temporal patterns for disease discrimination in cardiac echo videos. In N. Ayache, S. Ourselin, A. Maeder (Eds.), Lecture notes in computer science: Vol. 4791. Proc. 10th int. conf. medical image computing and computer assisted intervention (MICCAI), Brisbane, QLD, Australia (pp. 261–269). Berlin: Springer. Google Scholar
  43. Tenenbaum, J. B., & Freeman, W. T. (1996). Separating style and content. In M. Mozer, M. I. Jordan & T. Petsche (Eds.), Advances in neural information processing systems, Denver, CO, USA (pp. 662–668) 1996. Google Scholar
  44. Tenenbaum, J. B., & Freeman, W. T. (2000). Separating style and content with bilinear models. Neural Computation, 12(6), 1247–1283. CrossRefGoogle Scholar
  45. Turk, M., & Pentland, A. (1991). Eigenfaces for recognition. Journal of Cognitive Neuroscience, 3(1), 71–86. CrossRefGoogle Scholar
  46. Vasilescu, M. A. O., & Terzopoulos, D. (2003). Multilinear subspace analysis of image ensembles. In Proc. IEEE int. conf. on computer vision and pattern recognition (CVPR), Madison, WI, USA (pp. II: 93–99), 2006. Google Scholar

Copyright information

© Springer Science+Business Media, LLC 2009

Authors and Affiliations

  • Corné Hoogendoorn
    • 1
    • 2
  • Federico M. Sukno
    • 1
    • 2
  • Sebastián Ordás
    • 2
  • Alejandro F. Frangi
    • 1
    • 2
    Email author
  1. 1.Networking Biomedical Research Center on Bioengineering, Biomaterials and Nanomedicine (CIBER-BBN)BarcelonaSpain
  2. 2.Center for Computational Imaging and Simulation Technologies in Biomedicine (CISTIB), Department of Information and Communication TechnologiesUniversitat Pompeu FabraBarcelonaSpain

Personalised recommendations