From CMR Image to Patient-Specific Simulation and Population-Based Analysis: Tutorial for an Openly Available Image-Processing Pipeline

  • Maciej Marciniak
  • Hermenegild Arevalo
  • Jacob Tfelt-Hansen
  • Thomas Jespersen
  • Reza Jabbari
  • Charlotte Glinge
  • Kiril A. Ahtarovski
  • Niels Vejlstrup
  • Thomas Engstrom
  • Mary M. Maleckar
  • Kristin McLeod
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10124)


Cardiac magnetic resonance (CMR) imaging is becoming a routine diagnostic and therapy planning tool for some cardiovascular diseases. It is still challenging to properly analyse the acquired data, and the currently available measures do not exploit the rich characteristics of that data. Advanced analysis and modelling techniques are increasingly used to extract additional information from the images, in order to define metrics describing disease manifestations and to quantitatively compare patients. Many techniques share a common bottleneck caused by the image processing required to segment the images and convert the segmentation to a usable computational domain for analysis/modelling. To address this, we present a comprehensive pipeline to go from CMR images to computational bi-ventricle meshes. The latter can be used for biophysical simulations or statistical shape analysis. The provided tutorial describes each step and the proposed pipeline, which makes use of tools that are available open-source. The pipeline was applied to a data-set of myocardial infarction patients, from late gadolinium enhanced CMR images, to analyse and compare structure in these patients. Examples of applications present the use of the output of the pipeline for patient-specific biophysical simulations and population-based statistical shape analysis.

1 Introduction

Patient-specific simulations and population-based modelling are becoming increasingly popular in analyzing cardiac disease to extract important information about causes of arrhythmogenic or mechanical dysfunctions, and remodelling that occurs as a result of a disease. Such computational models have proven useful, for example, in detecting re-entry sites for arrhythmias [1] and to quantify structural abnormalities [2].

Computational patient-specific simulation requires segmentation of the anatomy from imaging. In the case of population-based modelling, pre-alignment of all subjects to a common frame is required, to remove differences in pose. For electrophysiology simulations of ischemic heart disease, segmentation of the scar or ischemic regions has to be performed. These regions are important pre-cursors in many arrhythmic disorders. Going from medical images to computational geometries can be time-consuming, laborious and prone to human error.

Since CMR imaging enables tissue characterisation through late gadolinium enhancement (LGE) imaging, the use of LGE is becoming increasingly popular. Due to the lack of openly available and automated tools, we developed a pipeline from LGE images to computational geometries that makes use of a freely available and widely used cardiac image segmentation software tool (Segment, Medviso). The proposed pipeline processes the segmentation to remove slice misalignment, to align all segmented data to a common reference (for comparative analysis and statistical shape modelling), and to build three-dimensional (3D) volumetric models of the geometry and scar regions (for electrophysiological simulation). A tutorial provides detailed steps of the proposed open-source pipeline and potential uses of the pipeline are described in our examples applied to myocardial infarction patients. 3D model generation and the techniques used to process the scar information are detailed in Sects. 2 and 3.

2 Bi-ventricle and Scar Segmentation

In the present work, 3D segmentation of the left and right ventricular endocardium, bi-ventricular epicardium, and scar (infarct) region from LGE images (in the end-diastolic phase) is performed. For this task, Segment; a software package for medical image analysis from Medviso [3], is used. A screenshot showing the functionality of Segment is given in Fig. 1. A full tutorial describing how to perform segmentation in Segment is provided on their website1.

Ventricle segmentation is accomplished by either manually or automatically drawing contours around the endocardium and epicardium surfaces of both ventricles on each two-dimensional (2D) image slice. Automatic segmentation within Segment is performed using level-set and deformable contour algorithms [4]. The algorithm creates both endo- and epicardial borders for the left ventricle (LV), which the user can adjust if necessary. Because of the complexity in shape of the right ventricle (RV), segmentation in Segment is performed semi-manually. In proposed pipeline, the tool for segmentation of the right ventricle epicardium is used to define the bi-ventricular epicardium. Segmentation is performed up to the last basal slice before the valve plane, defined as the first slice which divides into inlet and outlet of either ventricle. The bi-ventricular epicardium is required to generate a volumetric mesh for biophysical simulations. Sets of data points, which describe both epi- and endocardial borders of the ventricles, are saved from the image itself for each slice of the LGE stack.

The scar region is automatically delineated with an algorithm that also incorporates partial volume effects by weighting the infarct volume by pixel intensity [5]. The underlying algorithm for finding infarct is based on Expectation Maximization (EM algorithm). It is necessary to perform segmentation of the left ventricle first, as the ischemic regions are found between the endo- and epicardium of the LV. The method for computing scar in Segment is described in [6].
Fig. 1.

Annotated screenshot of the Segment user interface with highlighted main units, which shows functionality of the tool. Segmented ventricles with defined scar regions can be seen in the main viewing area.

3 Automatic Pre-processing Pipeline

3.1 3D Model Generation

From the slice-wise segmentation, a point-cloud defining the boundary (surface) points on each slice is exported. Acquired point-clouds are then processed in order to locate them in the same space [6]. Breath-hold artefact correction is achieved through calculating linear least square estimation of the vertical axis of the LV, according to the 2D barycenters of each slice. The barycenter of a given slice is then aligned to the slice above and the slice below. This is applied to point-clouds corresponding to both ventricles as well as to the scar regions.

Performing statistical shape analysis on meshes created in further steps requires both ventricles of every subject to be in the same physical space (i.e. to be rigidly aligned). Therefore, spatial alignment is performed to shift and rotate all subjects to a reference subject. This is necessary to reduce the bias in construction of the mean and to focus on calculating differences in the anatomy instead of position and/or orientation of given subjects. Alignment of all subjects is performed by taking the 3D barycenter of each ventricle from the endocardial point-clouds, and then a line-segment joining the LV and RV barycenters is computed. The rigid transformation from the line-segment of each subject to the line-segment of the arbitrarily chosen reference subject is computed. All point-clouds are then transformed by the computed line-segment transformation to align all 3D point-clouds. Translation parameters are based on the difference in positions of the barycenters in 3D space. Rotation is performed about the Z axis with angle \(\theta \) defined as (Eq. 1)
$$\begin{aligned} \theta = cos^{-1}\left( \frac{u{\cdot }v}{||u||\cdot ||v||}\right) \end{aligned}$$
with u and v being vectors joining the line segments. This yields the following rotation matrix (Eq. 2)
$$\begin{aligned} R_Z(\theta ) = \left[ \begin{array}{ccc} cos\theta &{} -sin\theta &{} 0\\ sin\theta &{} cos\theta &{} 0\\ 0 &{} 0 &{} 1 \end{array}\right] \end{aligned}$$
Once all patients are aligned to a common space, the volumetric 3D tetrahedral meshes are generated by combining the RV endocardium with the LV endo- and epicardium, and the bi-ventricular epicardium.

3.2 Scar Post-processing

A binary image of scar regions is used to calculate coordinates of scar points in 3D. For each subject, the scar regions are defined with a 3D boolean array, and are transformed along with the ventricular segmentation (slice correction and pairwise alignment). In order to retrieve the coordinates in the same space as the segmented and aligned ventricles, indices of the positions with boolean ‘one’ are found. These indices correspond to the X and Y position of the scar regions in each slice. Obtained coordinates are then adjusted to the image resolution. The third (Z) coordinate is found by inserting the voxel depth for each slice, equal to the gap between slices in the ventricles plus the slice thickness. The point-cloud is then shifted along the Z axis, to the location of the left ventricle.

Obtained point-clouds (defining the scar/ischemic zone) are processed in order to create a proper, uniform volume of the scar on which further analysis can be performed. This scar volume must also be projected to the ventricular volumetric space, to keep it in the correct location. For this purpose, several image processing tools have been applied to the scar data sets (see Fig. 2) to:
  • Eliminate outliers

  • Resample voxels to be isotropic

  • Smooth

  • Calculate iso-contours

The first step in the processing pipeline is performed to remove noise in the scar image using a median image filter, which eliminates non-physiological outliers. By applying the filter, every pixel/data point is replaced with the median of its neighbours. The value of the radius, defining the furthest neighbour of the pixel/data point in each direction, can be selected depending on the desired level of noise reduction.

The second step is to account for the large inter-slice spacing in the acquired images, which may result in loss of important information encoded in the third dimension (along the Z axis). In order to make the images useful for further steps of image analysis, re-sampling of the data-set is conducted. Resampling of the voxel size in the Z direction is performed to obtain voxel sizes equal to the X and Y directions (which are equal). This results in an isotropic 3D image, which provides more insights and information in the 3D space than rigid connection of slices. Thirdly, a Gaussian smoothing to remove the staircase-effect in the images is performed.

Finally, the iso-contours are calculated to allow for visual analysis. This is essentially achieved by extracting the surface of the scar. The used filter defines the borders of an object and uses them to create a 3D mesh. The extracted surface is surrounding the pixels that define the scar region. The created pipeline is summarized in Fig. 2.
Fig. 2.

The presented pipeline to go from a LGE image to point-clouds describing the endocardial surfaces of the left and right ventricles, the bi-ventricular epicardium, and scar regions (ventricle and scar segmentation), then to correct of slice misalignment caused by breath-holding (breathhold artefact correction), then to align subjects spatially to shift and rotate all subjects to a common space (spatial alignment), then to compute 3D scar volumes from the scar (ischemic) segmentation (scar extraction) and finally to post-processing of scar, which eliminates the outliers, and creates a 3D mesh (scar processing).

3.3 Implementation

As mentioned in Sect. 2, ventricular and scar segmentation was performed using Segment, a software package for medical image analysis from Medviso (FDA approved), freely available for research purposes. A licensed version is available for clinical use. The software not only allows segmentation with an easy-to-use interface, but also provides additional tools such as 3D visualisation of the obtained model, calculated volumes of both ventricles, and scar volume.

The described pre-processing pipeline steps were written in MATLAB (version R2015b). Point-clouds containing data describing the scar regions were saved as Insight Segmentation and Registration Toolkit (ITK)2 .mhd images using the Medical Image Processing Toolbox3. The volumetric mesh generation is performed using Gmsh4.

For the scar processing pipeline, the (ITK) was used which is an open-source, cross-platform system. Filters used for isotropic resampling, smoothing, and computing contours were written based on open-source examples5. Filtering was performed with shell script. Final results were visualised using Paraview from Kitware6. All codes used in this work are available open-source on Github repository7. Conversion of the Matlab code to python will be added in the near future to provide a pipeline that uses only open-source software.

4 Scar and Anatomy Analysis

4.1 Patient Data

The patient data used to develop the presented pipeline comes from an ongoing study in Denmark on genetic causes to ventricular arrhythmia in patients during first ST- elevation myocardial infarction (GEVAMI) [7]. LGE CMR images for these patients were collected from a retrospective database within approximately four weeks post-infarction period. A data-set of 8 patients, with varying extent of myocardial infarction, was studied: mean age ± standard deviation (years) \({=}59 \pm 9\). The final meshes for the two patients with the most significant ischemia are shown in Fig. 3, compared with 3D models of the segmentation from Segment. Slice misalignment, visible in 3D models, has been corrected for all patients. Infarct volumes were calculated using Simpsons rule [8].
Fig. 3.

The final meshes acquired with the pipeline in row B shown against the 3D model of the segmentation from Segment in row A. Presented patients have the greatest extent of ischemic zone.

Fig. 4.

(a) Example of a 3D heart model and tissue characterisation for one patient and (b) corresponding electrophysiology solution showing a reentrant circuit. Fiber directions were generated and integrated into the models using a rule-based algorithm.

4.2 Applications of the Presented Pipeline

To show how the presented pipeline can be used in practice, two examples are briefly described; one for patient-specific analysis and one for population-based modelling. The first example uses the volumetric mesh and scar segmentation to perform patient-specific electrophysiology simulation. The second example used the aligned surfaces to perform population-based statistical shape analysis.

Patient-Specific Electrophysiology Simulation: The presented pipeline was used to extract full 3D heart models with tissue characterisation (healthy/ischemic) for electrophysiology simulations on the MI patient data-set. A mono-domain electrophysiology model was used, modelling ischemic regions as having reduced conductivity, as described in [9]. An example of the electrophysiology solution for one of these patients is shown in Fig. 4. The electrophysiology simulations were run using the CARP software8 (licensed software).

Population-Based Ventricular Shape Analysis: Statistical shape analysis was applied to 8 patients to compute the most common shape features in the population (i.e. the shape modes). The ventricular surfaces were extracted and aligned using the presented pipeline. Principal component analysis (PCA) was used to compute the shape modes, following the methods described in [10]. Four shapes (modes) captured 90% of the shape variance in the population (see Fig. 5). The methods used to perform the statistical shape analysis are available open-source9.
Fig. 5.

The first four PCA shape modes (accounting for 90% of the shape variability in the population) are shown from low values (\({-}2\) standard deviations, bottom row), to high values (\({+}2\) standard deviations, top row), with the percentage of shape variance described by each shape given in brackets. Arrows highlight the important regions where changes are visible.

5 Discussion

This study describes the pipeline to go from CMR images to bi-ventricular meshes with scar and infarct regions extracted, with results applicable for both simulations and analysis. There have been a number of methods proposed for segmentation, scar extraction, image processing, and mesh generation, but only some of them are openly available. The tools used in the proposed pipeline were chosen for their usability, applicability, and availability. Our objective is not to create the optimal pipeline (given that gold standards do not exist and applications are varied), but rather to create a pipeline that could be easily adapted to other segmentation/scar extraction/mesh generation tools, and easily applicable to other modelling and analysis applications. There are many limitations to the currently available tools and methods, which are discussed below and will be addressed in future releases.

Tissue Characterisation: In the created pipeline, LGE was used for locating ischemic regions. It was considered in this study because despite its limitations and occurring variability of signal-to-noise ratio, LGE is still the most commonly used biomarker for tissue characterisation among clinicians. A different approach, such as T1 mapping, could be used for accurate in-vivo identification of fibrosis as an alternative, but is not widely used in clinical practice for scar quantification. More robust tools for scar detection are continuously being developed (e.g. those submitted to the 2012 STACOM challenge10) and might be of higher accuracy and wider use in the future.

Segmentation Method: Segmentation was performed using Segment, which was chosen for the presented pipeline due to the accessibility and easy-to-use interface. What’s more, it contains all the information needed to calculate the volumes of the scar and both ventricles, show weighted infarct transmurality, and visualise the pre-processed model. These utilities, contained in one software tool, comply with the requirements for full pipeline generation. The daily download rate of Segment is 5–6 downloads (including upgrades), with the number of unique new users per year reaching around 1500 (those that use unique email addresses). In addition, 17 clinical hospitals around the world use the commercial version of Segment (Segment CMR). To date, there are close to 600 journal publications that reference Segment. Therefore, it is a widely used tool and because it is continually improved with new methods as they become available, it will likely continue to be used for both research and clinical purposes. Potential expansion would include choosing different segmentation algorithms, such as those proposed in the STACOM 2011 challenge11. Furthermore, other open-source tools, such as Slicer12, could have been used for segmentation and will be investigated. The proposed pipeline was applied to LGE images, but in the case where tissue classification is not required, the same pipeline could be applied e.g. to cine CMR images or T1 images.

Segmentation Area: In this study, valve segmentation is omitted due to the nature of exemplary applications. In general, statistical shape analysis and patient-specific electrophysiology simulations are conducted on computational ventricle models segmented from the frame below the valve plane. In future studies, valve modelling will be included using methods such as the one described in [11] to further increase the usability, provided that such tools become openly available.

Alignment and Correction: Slice correction and spatial alignment were both performed rigidly (i.e. no stretch or shear). Slice misalignment is not necessarily a rigid translation of the image slice, and could indeed include non-rigid transformation depending on the how consistently the patient held their breath for each scan. For simplicity, only rigid translation was considered in this pipeline, i.e. by calculating the vector joining the barycentre of each ventricle. Other rigid alignment techniques that consider the full geometry could be used, such as the robust point-set registration algorithm using Gaussian Mixture Models13.

Image Processing Tool: In order to process the scar images, filter them, and compute contours, ITK was used. ITK is a cross-platform, open-source tool, which provides great functionality. Moreover, it is well documented and numerous examples of use are published on the official website. Filters and functions are constantly in development and added by independent users. ITK is the most commonly used tool for processing medical images, which was the main criterion for the choice of software in this study. Other tools for computational mesh generation, such as CGAL14 or iso2mesh15 could have been used as well and can be easily adopted in lieu of the tools used in the proposed pipeline.

Applications: Two potential applications were described: patient-specific biophysical simulation and population-based statistical shape analysis. The former is conducted with CARP software, which is only available under license. Although the purpose of describing this simulation was to show the applicability of the output of the pipeline, it is worth mentioning that open source tools, such as Fenics16, ECGSim17, Chaste18 or CellML19, or different methods [12] could be used for similar simulations. In addition, other statistical shape analyses could be applied to the computational ventricle meshes, for example those applied to the STACOM 2015 challenge20.

6 Conclusion

A full pipeline from LGE CMR images to aligned computational surface meshes for population-based modelling or volumetric meshes with scar information for patient-specific biophysical simulations is presented. The pipeline includes steps to correct for breath-hold misalignment common in CMR images, to align all subjects in a common space, to process scar information to obtain physiological volumetric models of scar, and to build 3D models of both the ventricles and the scar regions. A tutorial outlining how to apply the pipeline, which makes use of open-source segmentation and image and mesh processing, is provided. Examples of potential applications of the pipeline to perform electrophysiology simulations or statistical shape modelling in myocardial infarction patients are given. All code is available open-source on Github repository, for use by other researchers.




This project was partially funded by the Centre for Cardiological Innovation (CCI), Norway funded by the Norwegian Research Council, and Novo Nordic foundation.


  1. 1.
    Arevalo, H.J., Vadakkumpadan, F., Guallar, E., Jebb, A., Malamas, P., Wu, K.C., Trayanova, N.A.: Arrhythmia risk stratification of patients after myocardial infarction using personalized heart models. Nat. Commun. 7 (2016)Google Scholar
  2. 2.
    Zhang, X., Cowan, B.R., Bluemke, D.A., Finn, J.P., Fonseca, C.G., Kadish, A.H., Lee, D.C., Lima, J.A., Suinesiaputra, A., Young, A.A., et al.: Atlas-based quantification of cardiac remodeling due to myocardial infarction. PLoS One 9(10), e110243 (2014)CrossRefGoogle Scholar
  3. 3.
    Heiberg, E., Sjgren, J., Ugander, M., Carlsson, M., Engblom, H., Arheden, H.: Design and validation of segment-freely available software for cardiovascular image analysis. BMC Med. Imaging 10(1) (2010)Google Scholar
  4. 4.
    Heiberg, E., Wigstrom, L., Carlsson, M., Bolger, A., Karlsson, M.: Time resolved three-dimensional automated segmentation of the left ventricle. In: Computers in Cardiology, 2005, pp. 599–602. IEEE (2005)Google Scholar
  5. 5.
    Engblom, H., Tufvesson, J., Jablonowski, R., Carlsson, M., Aletras, A.H., Hoffmann, P., Jacquier, A., Kober, F., Metzler, B., Erlinge, D., et al.: A new automatic algorithm for quantification of myocardial infarction imaged by late gadolinium enhancement cardiovascular magnetic resonance: experimental validation and comparison to expert delineations in multi-center, multi-vendor patient data. J. Cardiovasc. Magn. Reson. 18(1), 1 (2016)CrossRefGoogle Scholar
  6. 6.
    Heiberg, E., Ugander, M., Engblom, H., Gotberg, M., Olivecrona, G.K., Erlinge, D., Arheden, H.: Automated quantification of myocardial infarction from MR images by accounting for partial volume effects: animal, phantom, and human study 1. Radiology 246(2), 581–588 (2008)CrossRefGoogle Scholar
  7. 7.
    Jabbari, R., Engstrøm, T., Glinge, C., Risgaard, B., Jabbari, J., Winkel, B.G., Terkelsen, C.J., Tilsted, H.H., Jensen, L.O., Hougaard, M., et al.: Incidence and risk factors of ventricular fibrillation before primary angioplasty in patients with first st-elevation myocardial infarction: a nationwide study in Denmark. J. Am. Heart Assoc. 4(1), e001399 (2015)CrossRefGoogle Scholar
  8. 8.
    Hergan, K., Schuster, A., Fruhwald, J., Mair, M., Burger, R., Topker, M.: Comparison of left and right ventricular volume measurement using the Simpson’s method and the area length method. Eur. J. Radiol. 65(2), 270–278 (2008)CrossRefGoogle Scholar
  9. 9.
    Arevalo, H., Helm, P., Trayanova, N.: Development of a model of the infarcted canine heart that predicts arrhythmia generation from specific cardiac geometry and scar distribution. In: Computers in Cardiology. IEEE 2008, pp. 497–500 (2008)Google Scholar
  10. 10.
    Durrleman, S., Pennec, X., Trouvé, A., Ayache, N.: Statistical models of sets of curves and surfaces based on currents. Med. Image Anal. 13, 793–808 (2009)CrossRefGoogle Scholar
  11. 11.
    Gilbert, K., Lam, H.I., Pontré, B., Cowan, B., Occleshaw, C., Liu, J., Young, A.: An interactive tool for rapid biventricular analysis of congenital heart disease. Clin. Physiol. Funct. Imaging (2015)Google Scholar
  12. 12.
    Pop, M., et al.: EP challenge - STACOM’11: forward approaches to computational electrophysiology using MRI-based models and in-vivo CARTO mapping in swine hearts. In: Camara, O., Konukoglu, E., Pop, M., Rhode, K., Sermesant, M., Young, A. (eds.) STACOM 2011. LNCS, vol. 7085, pp. 1–13. Springer, Heidelberg (2012). doi:10.1007/978-3-642-28326-0_1 CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Maciej Marciniak
    • 1
  • Hermenegild Arevalo
    • 1
  • Jacob Tfelt-Hansen
    • 2
  • Thomas Jespersen
    • 3
  • Reza Jabbari
    • 2
  • Charlotte Glinge
    • 2
  • Kiril A. Ahtarovski
    • 2
  • Niels Vejlstrup
    • 2
  • Thomas Engstrom
    • 2
  • Mary M. Maleckar
    • 1
  • Kristin McLeod
    • 1
    • 4
  1. 1.Cardiac Modelling DepartmentSimula Research LaboratoryOsloNorway
  2. 2.Department of CardiologyRigshospitaletCopenhagenDenmark
  3. 3.Department of Biomedical SciencesUniversity of CopenhagenCopenhagenDenmark
  4. 4.Centre for Cardiological InnovationOsloNorway

Personalised recommendations