From CMR Image to Patient-Specific Simulation and Population-Based Analysis: Tutorial for an Openly Available Image-Processing Pipeline
Cardiac magnetic resonance (CMR) imaging is becoming a routine diagnostic and therapy planning tool for some cardiovascular diseases. It is still challenging to properly analyse the acquired data, and the currently available measures do not exploit the rich characteristics of that data. Advanced analysis and modelling techniques are increasingly used to extract additional information from the images, in order to define metrics describing disease manifestations and to quantitatively compare patients. Many techniques share a common bottleneck caused by the image processing required to segment the images and convert the segmentation to a usable computational domain for analysis/modelling. To address this, we present a comprehensive pipeline to go from CMR images to computational bi-ventricle meshes. The latter can be used for biophysical simulations or statistical shape analysis. The provided tutorial describes each step and the proposed pipeline, which makes use of tools that are available open-source. The pipeline was applied to a data-set of myocardial infarction patients, from late gadolinium enhanced CMR images, to analyse and compare structure in these patients. Examples of applications present the use of the output of the pipeline for patient-specific biophysical simulations and population-based statistical shape analysis.
Patient-specific simulations and population-based modelling are becoming increasingly popular in analyzing cardiac disease to extract important information about causes of arrhythmogenic or mechanical dysfunctions, and remodelling that occurs as a result of a disease. Such computational models have proven useful, for example, in detecting re-entry sites for arrhythmias  and to quantify structural abnormalities .
Computational patient-specific simulation requires segmentation of the anatomy from imaging. In the case of population-based modelling, pre-alignment of all subjects to a common frame is required, to remove differences in pose. For electrophysiology simulations of ischemic heart disease, segmentation of the scar or ischemic regions has to be performed. These regions are important pre-cursors in many arrhythmic disorders. Going from medical images to computational geometries can be time-consuming, laborious and prone to human error.
Since CMR imaging enables tissue characterisation through late gadolinium enhancement (LGE) imaging, the use of LGE is becoming increasingly popular. Due to the lack of openly available and automated tools, we developed a pipeline from LGE images to computational geometries that makes use of a freely available and widely used cardiac image segmentation software tool (Segment, Medviso). The proposed pipeline processes the segmentation to remove slice misalignment, to align all segmented data to a common reference (for comparative analysis and statistical shape modelling), and to build three-dimensional (3D) volumetric models of the geometry and scar regions (for electrophysiological simulation). A tutorial provides detailed steps of the proposed open-source pipeline and potential uses of the pipeline are described in our examples applied to myocardial infarction patients. 3D model generation and the techniques used to process the scar information are detailed in Sects. 2 and 3.
2 Bi-ventricle and Scar Segmentation
In the present work, 3D segmentation of the left and right ventricular endocardium, bi-ventricular epicardium, and scar (infarct) region from LGE images (in the end-diastolic phase) is performed. For this task, Segment; a software package for medical image analysis from Medviso , is used. A screenshot showing the functionality of Segment is given in Fig. 1. A full tutorial describing how to perform segmentation in Segment is provided on their website1.
Ventricle segmentation is accomplished by either manually or automatically drawing contours around the endocardium and epicardium surfaces of both ventricles on each two-dimensional (2D) image slice. Automatic segmentation within Segment is performed using level-set and deformable contour algorithms . The algorithm creates both endo- and epicardial borders for the left ventricle (LV), which the user can adjust if necessary. Because of the complexity in shape of the right ventricle (RV), segmentation in Segment is performed semi-manually. In proposed pipeline, the tool for segmentation of the right ventricle epicardium is used to define the bi-ventricular epicardium. Segmentation is performed up to the last basal slice before the valve plane, defined as the first slice which divides into inlet and outlet of either ventricle. The bi-ventricular epicardium is required to generate a volumetric mesh for biophysical simulations. Sets of data points, which describe both epi- and endocardial borders of the ventricles, are saved from the image itself for each slice of the LGE stack.
3 Automatic Pre-processing Pipeline
3.1 3D Model Generation
From the slice-wise segmentation, a point-cloud defining the boundary (surface) points on each slice is exported. Acquired point-clouds are then processed in order to locate them in the same space . Breath-hold artefact correction is achieved through calculating linear least square estimation of the vertical axis of the LV, according to the 2D barycenters of each slice. The barycenter of a given slice is then aligned to the slice above and the slice below. This is applied to point-clouds corresponding to both ventricles as well as to the scar regions.
3.2 Scar Post-processing
A binary image of scar regions is used to calculate coordinates of scar points in 3D. For each subject, the scar regions are defined with a 3D boolean array, and are transformed along with the ventricular segmentation (slice correction and pairwise alignment). In order to retrieve the coordinates in the same space as the segmented and aligned ventricles, indices of the positions with boolean ‘one’ are found. These indices correspond to the X and Y position of the scar regions in each slice. Obtained coordinates are then adjusted to the image resolution. The third (Z) coordinate is found by inserting the voxel depth for each slice, equal to the gap between slices in the ventricles plus the slice thickness. The point-cloud is then shifted along the Z axis, to the location of the left ventricle.
Resample voxels to be isotropic
The first step in the processing pipeline is performed to remove noise in the scar image using a median image filter, which eliminates non-physiological outliers. By applying the filter, every pixel/data point is replaced with the median of its neighbours. The value of the radius, defining the furthest neighbour of the pixel/data point in each direction, can be selected depending on the desired level of noise reduction.
The second step is to account for the large inter-slice spacing in the acquired images, which may result in loss of important information encoded in the third dimension (along the Z axis). In order to make the images useful for further steps of image analysis, re-sampling of the data-set is conducted. Resampling of the voxel size in the Z direction is performed to obtain voxel sizes equal to the X and Y directions (which are equal). This results in an isotropic 3D image, which provides more insights and information in the 3D space than rigid connection of slices. Thirdly, a Gaussian smoothing to remove the staircase-effect in the images is performed.
As mentioned in Sect. 2, ventricular and scar segmentation was performed using Segment, a software package for medical image analysis from Medviso (FDA approved), freely available for research purposes. A licensed version is available for clinical use. The software not only allows segmentation with an easy-to-use interface, but also provides additional tools such as 3D visualisation of the obtained model, calculated volumes of both ventricles, and scar volume.
The described pre-processing pipeline steps were written in MATLAB (version R2015b). Point-clouds containing data describing the scar regions were saved as Insight Segmentation and Registration Toolkit (ITK)2 .mhd images using the Medical Image Processing Toolbox3. The volumetric mesh generation is performed using Gmsh4.
For the scar processing pipeline, the (ITK) was used which is an open-source, cross-platform system. Filters used for isotropic resampling, smoothing, and computing contours were written based on open-source examples5. Filtering was performed with shell script. Final results were visualised using Paraview from Kitware6. All codes used in this work are available open-source on Github repository7. Conversion of the Matlab code to python will be added in the near future to provide a pipeline that uses only open-source software.
4 Scar and Anatomy Analysis
4.1 Patient Data
4.2 Applications of the Presented Pipeline
To show how the presented pipeline can be used in practice, two examples are briefly described; one for patient-specific analysis and one for population-based modelling. The first example uses the volumetric mesh and scar segmentation to perform patient-specific electrophysiology simulation. The second example used the aligned surfaces to perform population-based statistical shape analysis.
Patient-Specific Electrophysiology Simulation: The presented pipeline was used to extract full 3D heart models with tissue characterisation (healthy/ischemic) for electrophysiology simulations on the MI patient data-set. A mono-domain electrophysiology model was used, modelling ischemic regions as having reduced conductivity, as described in . An example of the electrophysiology solution for one of these patients is shown in Fig. 4. The electrophysiology simulations were run using the CARP software8 (licensed software).
This study describes the pipeline to go from CMR images to bi-ventricular meshes with scar and infarct regions extracted, with results applicable for both simulations and analysis. There have been a number of methods proposed for segmentation, scar extraction, image processing, and mesh generation, but only some of them are openly available. The tools used in the proposed pipeline were chosen for their usability, applicability, and availability. Our objective is not to create the optimal pipeline (given that gold standards do not exist and applications are varied), but rather to create a pipeline that could be easily adapted to other segmentation/scar extraction/mesh generation tools, and easily applicable to other modelling and analysis applications. There are many limitations to the currently available tools and methods, which are discussed below and will be addressed in future releases.
Tissue Characterisation: In the created pipeline, LGE was used for locating ischemic regions. It was considered in this study because despite its limitations and occurring variability of signal-to-noise ratio, LGE is still the most commonly used biomarker for tissue characterisation among clinicians. A different approach, such as T1 mapping, could be used for accurate in-vivo identification of fibrosis as an alternative, but is not widely used in clinical practice for scar quantification. More robust tools for scar detection are continuously being developed (e.g. those submitted to the 2012 STACOM challenge10) and might be of higher accuracy and wider use in the future.
Segmentation Method: Segmentation was performed using Segment, which was chosen for the presented pipeline due to the accessibility and easy-to-use interface. What’s more, it contains all the information needed to calculate the volumes of the scar and both ventricles, show weighted infarct transmurality, and visualise the pre-processed model. These utilities, contained in one software tool, comply with the requirements for full pipeline generation. The daily download rate of Segment is 5–6 downloads (including upgrades), with the number of unique new users per year reaching around 1500 (those that use unique email addresses). In addition, 17 clinical hospitals around the world use the commercial version of Segment (Segment CMR). To date, there are close to 600 journal publications that reference Segment. Therefore, it is a widely used tool and because it is continually improved with new methods as they become available, it will likely continue to be used for both research and clinical purposes. Potential expansion would include choosing different segmentation algorithms, such as those proposed in the STACOM 2011 challenge11. Furthermore, other open-source tools, such as Slicer12, could have been used for segmentation and will be investigated. The proposed pipeline was applied to LGE images, but in the case where tissue classification is not required, the same pipeline could be applied e.g. to cine CMR images or T1 images.
Segmentation Area: In this study, valve segmentation is omitted due to the nature of exemplary applications. In general, statistical shape analysis and patient-specific electrophysiology simulations are conducted on computational ventricle models segmented from the frame below the valve plane. In future studies, valve modelling will be included using methods such as the one described in  to further increase the usability, provided that such tools become openly available.
Alignment and Correction: Slice correction and spatial alignment were both performed rigidly (i.e. no stretch or shear). Slice misalignment is not necessarily a rigid translation of the image slice, and could indeed include non-rigid transformation depending on the how consistently the patient held their breath for each scan. For simplicity, only rigid translation was considered in this pipeline, i.e. by calculating the vector joining the barycentre of each ventricle. Other rigid alignment techniques that consider the full geometry could be used, such as the robust point-set registration algorithm using Gaussian Mixture Models13.
Image Processing Tool: In order to process the scar images, filter them, and compute contours, ITK was used. ITK is a cross-platform, open-source tool, which provides great functionality. Moreover, it is well documented and numerous examples of use are published on the official website. Filters and functions are constantly in development and added by independent users. ITK is the most commonly used tool for processing medical images, which was the main criterion for the choice of software in this study. Other tools for computational mesh generation, such as CGAL14 or iso2mesh15 could have been used as well and can be easily adopted in lieu of the tools used in the proposed pipeline.
Applications: Two potential applications were described: patient-specific biophysical simulation and population-based statistical shape analysis. The former is conducted with CARP software, which is only available under license. Although the purpose of describing this simulation was to show the applicability of the output of the pipeline, it is worth mentioning that open source tools, such as Fenics16, ECGSim17, Chaste18 or CellML19, or different methods  could be used for similar simulations. In addition, other statistical shape analyses could be applied to the computational ventricle meshes, for example those applied to the STACOM 2015 challenge20.
A full pipeline from LGE CMR images to aligned computational surface meshes for population-based modelling or volumetric meshes with scar information for patient-specific biophysical simulations is presented. The pipeline includes steps to correct for breath-hold misalignment common in CMR images, to align all subjects in a common space, to process scar information to obtain physiological volumetric models of scar, and to build 3D models of both the ventricles and the scar regions. A tutorial outlining how to apply the pipeline, which makes use of open-source segmentation and image and mesh processing, is provided. Examples of potential applications of the pipeline to perform electrophysiology simulations or statistical shape modelling in myocardial infarction patients are given. All code is available open-source on Github repository, for use by other researchers.
This project was partially funded by the Centre for Cardiological Innovation (CCI), Norway funded by the Norwegian Research Council, and Novo Nordic foundation.
- 1.Arevalo, H.J., Vadakkumpadan, F., Guallar, E., Jebb, A., Malamas, P., Wu, K.C., Trayanova, N.A.: Arrhythmia risk stratification of patients after myocardial infarction using personalized heart models. Nat. Commun. 7 (2016)Google Scholar
- 3.Heiberg, E., Sjgren, J., Ugander, M., Carlsson, M., Engblom, H., Arheden, H.: Design and validation of segment-freely available software for cardiovascular image analysis. BMC Med. Imaging 10(1) (2010)Google Scholar
- 4.Heiberg, E., Wigstrom, L., Carlsson, M., Bolger, A., Karlsson, M.: Time resolved three-dimensional automated segmentation of the left ventricle. In: Computers in Cardiology, 2005, pp. 599–602. IEEE (2005)Google Scholar
- 5.Engblom, H., Tufvesson, J., Jablonowski, R., Carlsson, M., Aletras, A.H., Hoffmann, P., Jacquier, A., Kober, F., Metzler, B., Erlinge, D., et al.: A new automatic algorithm for quantification of myocardial infarction imaged by late gadolinium enhancement cardiovascular magnetic resonance: experimental validation and comparison to expert delineations in multi-center, multi-vendor patient data. J. Cardiovasc. Magn. Reson. 18(1), 1 (2016)CrossRefGoogle Scholar
- 7.Jabbari, R., Engstrøm, T., Glinge, C., Risgaard, B., Jabbari, J., Winkel, B.G., Terkelsen, C.J., Tilsted, H.H., Jensen, L.O., Hougaard, M., et al.: Incidence and risk factors of ventricular fibrillation before primary angioplasty in patients with first st-elevation myocardial infarction: a nationwide study in Denmark. J. Am. Heart Assoc. 4(1), e001399 (2015)CrossRefGoogle Scholar
- 9.Arevalo, H., Helm, P., Trayanova, N.: Development of a model of the infarcted canine heart that predicts arrhythmia generation from specific cardiac geometry and scar distribution. In: Computers in Cardiology. IEEE 2008, pp. 497–500 (2008)Google Scholar
- 11.Gilbert, K., Lam, H.I., Pontré, B., Cowan, B., Occleshaw, C., Liu, J., Young, A.: An interactive tool for rapid biventricular analysis of congenital heart disease. Clin. Physiol. Funct. Imaging (2015)Google Scholar
- 12.Pop, M., et al.: EP challenge - STACOM’11: forward approaches to computational electrophysiology using MRI-based models and in-vivo CARTO mapping in swine hearts. In: Camara, O., Konukoglu, E., Pop, M., Rhode, K., Sermesant, M., Young, A. (eds.) STACOM 2011. LNCS, vol. 7085, pp. 1–13. Springer, Heidelberg (2012). doi:10.1007/978-3-642-28326-0_1 CrossRefGoogle Scholar