Modelling Respiration Induced Torso Deformation Using a Mesh Fitting Algorithm

Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10117)


Precise positioning of an ablation probe in soft abdominal organs requires taking the respiration effects into account. Fast and reliable registration of a virtual abdominal organ with intra-operational imaging data remains a challenge in image-guided and Virtual Reality (VR) aided surgeries. In this paper we present a Host Mesh Fitting (HMF) algorithm to imitate the deformation of a torso due to aspiration effects. Displacements of the torso mesh are driven by virtual fiducial markers placed on the abdominal surface, which consequently deform abdominal organs in an implicit manner and with a small computational cost. In order to test the HMF algorithm a gelatine phantom was made with its internal channels detectable from ultrasonic imaging. Deformation of the channels due to a compression force was reproduced from the warping of the host mesh. After coupling with a fiducial marker tracking system the HMF algorithm can be used to model the torso deformation due to respiration effects.


Virtual abdominal organ Host Mesh Fitting Respiration effects Fiducial markers 

1 Introduction

Percutaneous minimally invasive procedures have become alternatives to traditional open surgeries due to their lower complication rates, shorter hospital stays and less expenses [1]. These procedures are usually aided by pre-operational computed tomography (CT) or magnetic resonance imaging (MRI) scans and guided by intra-operational imaging data, e.g., from cone beam CT and/or ultrasonic images [1]. Since the respiration causes abdominal organ displacements, virtual models built from CT/MRI images need to be registered with intra-operational data by respiration gated or breathing holding techniques, so that an intervention procedure can be performed at the same phase as that of the CT/MRI scan [2]. An alternative solution is 4D CT where multiple images are acquired during the rotation of the CT gantry, and the patient’s respiration is monitored by an exterior marker attached to the patient’s abdomen [2, 3]. As 4D CT introduces more radiation to patients and operators, computer algorithms have been proposed to interpolate image volume between the expiration and inspiration phases, thus the radiation dose can be reduced [2, 4].

Among the computational interpolation schemes, a deformable, B-Spline based registration model is proposed in [2] where the contour of an organ is automatically mapped from one phase to another, and a mapping accuracy of 3 mm is achieved. In [4], the position of a tumour is estimated using a nonlinear registration algorithm and its new position is reproduced by image morphing.

In this paper we propose an approach based on a Host Mesh Fitting (HMF) algorithm. The core of the algorithm is two sets of finite element mesh, i.e. the slave and the host mesh, whereby the deformation of the host mesh drives the motion of the slave mesh. This algorithm has been described in literatures, e.g., in [5] for muscle modelling. In this paper we extend this algorithm to the context of image warping as analogous to CT image interpolation of [4]. This is achieved by incorporating image voxels into the host mesh, therefore a volume image can be morphed between difference breathing phases. The concept is illustrated in Fig. 1, where the image intensities of a stack of CT images are normalised into the texture space (0, 1), and placed into a cube-shaped host mesh. By applying the HMF algorithm [5], the texture space is morphed after altering the direction vectors and positions of the nodes of the host mesh.
Fig. 1.

The concept of 3D image morphing within a finite element mesh: (a) A 3D image volume is placed in a tricubic Hermite element; and (b) arbitrary morphing can be made by adjusting the orientation or location of mesh nodes.

In the following sections, we will outline the HMF algorithm and describe its application in a CT image containing a human torso. Since respiration-gated CT data are not available for validation, we will use a gelatine phantom to compare the displacements simulated from the HMF algorithm and that derived from ultrasound imaging. The main goal is to elaborate the fitting algorithm with fiducial markers. The visualisation tool used in the work is CMGUI (, an open source visualisation and imaging software.

2 Methods

2.1 Host Mesh Fitting

The host-mesh-fitting algorithm is a subset of the free-form deformation technique [5]. The idea is to deform an object by enclosing it within a bounding object, and by deforming the bounding object the enclosed object will be deformed accordingly. The concept is illustrated in Fig. 2, where a surface mesh (the slave mesh) is completely enclosed within a 3D host mesh (thus the name). Since the local coordinates \((\eta _{1}, \eta _{2}, \eta _{3})\) of every node of the slave mesh can be written as a function of the coordinates of the host mesh \((\xi _{1}, \xi _{2}, \xi _{3})\) and the relative nodal positions of the slave mesh with respect to the host mesh remain intact, when the host mesh is deformed the slave mesh will be updated accordingly.
Fig. 2.

Description of a host-mesh and a slave mesh and their respective coordinate systems. A surface mesh i.e. the slave mesh is completely enclosed within a 3D host mesh.

Parametric representation of the host and slave mesh has been described in many literatures. In particular, we refer the interested reader to [5] for mathematical details of the fitting algorithm. We should highlight that the host mesh can be viewed as a Finite Element mesh, and any internal points within the mesh can be expressed as a weighted sum of basis functions. It should also be stressed that although the slave mesh in Fig. 2 is a 2D surface patch, a more complex mesh made of tetrahedra can also be used. Indeed a tetrehedra mesh will be used to simulate the deformation of a phantom (described later in Sect. 2.3). Furthermore, the slave mesh can be multi-dimensional, i.e., containing a combination of 1D, 2D and 3D mesh of vasculature, surface and parenchyma of an organ as introduced in [6].

2.2 Meshing an Image Volume

In the image morphing example of Fig. 1, the host mesh is a tricubic Hermite element, i.e. any point X in it can be expressed as \(X=\sum \phi _{i}\xi _{j}\) where \(\phi \) is the basis function (in this case a Hermite function) and \(\xi \) the nodal coordinates. In order to model soft organs, a more sophisticated host mesh needs to be used to represent their respective locations in the texture space, so that their deformation can be simulated. For example, bones are rigid and therefore they cannot be morphed in the same way as elastic tissues/organs. This concept is illustrated in Fig. 3 where the vertebra (indicated by an arrow) are separated from the soft abdominal cavity.
Fig. 3.

A CT image containing a human torso and the Finite Element host mesh is constructed for the volume image: (a) a trilinear mesh (\(8\times 8 \times 8\)) is used as the host mesh; and (b) a custom-made mesh arranged around the torso surface. The arrow indicates the location of vertebra which are not contained in the elastic torso mesh.

In Fig. 3(a) the host mesh is made of a trilinear mesh of 128 elements which are of the same size. The advantage of this method is that minimum efforts are required to create the host mesh, however the mesh does not define any specific organs, which have to be differentiated by an image segmentation algorithm. In Fig. 3(b) a custom-made host mesh was constructed around the torso surface so that its deformation can be simulated. This method has the advantage of being able to describe deformation of individual organs/tissues, but also bears the disadvantage of a high-cost in the mesh construction process.

In order to morph the host mesh, a set of source points within the mesh and their corresponding target points are fitted using a least square quasi Newton method [7] within OPT++, an open source library for nonlinear optimization algorithms.

Only four key landmark or fiducial markers are chosen as the source points, two on the diaphragm apexes and the other two on the surface of the chest (shown in Fig. 5 of the Results section). According to [1, 8], the inferior-superior movement of the diaphragm apex is \(27.3 \pm 10.2\,\text {mm}\), and the anterior-posterior translation of abdominal organs is around 8 to 10 mm. Thus a set of target points can be prescribed to guide the motion of the torso mesh. This feature is valuable when physical landmarks are not available on the torso, which is the case for this preliminary study.

2.3 Gelatine Phantom and Ultrasonic Imaging

Since respiratory gating CT images were not available for this study, a phantom was used to validate the HMF algorithm. The aim was to check the accuracy of simulated deformation of the phantom, in comparison with the ultrasound imaging data. The phantom (67 mm \(\times \) 92 mm \(\times \) 54 mm), shown in Fig. 4, was made of gelatine, a commonly used material for bio-tissues [9]. Softness of the phantom can be controlled by using different percentage of gelatine powder mixed with water. In this specific phantom 23% of gelatine power was used. Meanwhile two perpendicular tubes (diameter 10 mm) were placed in the water container. After the gelatine mixture was solidified the two tubes were removed forming two hollow channels, as shown in Fig. 4.
Fig. 4.

Left: the phantom contains two perpendicularly arranged channels and is compressed by forces applied laternally; Right: the transducer of Voluson i BT14 was used for ultrasound scanning.

In the actual ultrasonic scan experiment, the hollow channels were filled with water so that there was no air inside the phantom to interfere with ultrasound signals. After a compression force was applied from lateral sides of the phantom, the channels were deformed and scanned.

We used a Voluson i BT14 (GE) scanner in its B-mode to collect real-time ultrasonic imaging data from the compressed phantom, as analogous to the respiration cycle experienced by a torso. We also created a virtual computer model for the phantom, shown in Fig. 8 of the Results section, which consists of 4,925 tetrahedra elements. By fitting the ultrasound image into the virtual model the HMF algorithm can be tested.

3 Results

3.1 Simulation of Respiration Effects on Torso

A simulation was made for the respiration effects on the torso using the method described above. As mentioned earlier the data for the abdominal organ movements was adopted from [3, 8] and prescribed to the source points. In Fig. 5(a), the torso was assumed to be at the end of the expiration phase and the red markers, representing the source points, are at their most posterior positions. The golden arrows represent trajectory vectors from the source points to their corresponding target points. The additional two markers are placed on the apex of the diaphragm, and are hidden behind the iso-surface of the torso. In Fig. 5(b), the minimization and smoothing problem was solved in a desktop computer (Intel i7-4790 CPU @ 3.60 GHz, RAM 32 GB, GeForce 2 GB GTX 745 GPU). The computation took 0.7 s to complete.
Fig. 5.

A simulation of respiration effects on torso. (a) The torso at a quiet breath holding state after expiration; and (b) Simulated torso position at the start of expiration (or end of inspiration) after applying the HMF algorithm. Note the red points are the set of source points and the arrows are the projection vectors from the source points to target points. Also note the deformed host mesh, which drives the slave torso mesh. (Color figure online)

Fig. 6.

Host mesh fitting algorithm applied to torso deformation due to respiration. The arrows indicate mesh displacements occurring at the anterior-posterior perspective. (Color figure online)

Figure 6 yields more information about the fitting results. The black arrows indicate the anterior-posterior displacement of the torso surface. Meanwhile the image volume warps between respiration phases. This is shown from the three cross sections of the image, where the blue colour stands for the air and the light blue colour represents tissue. Also note the host mesh, i.e. the cube deforms during the fitting process, which in turn drove the torso displacements.

3.2 Experiments with the Phantom

The phantom shown in Fig. 4 was used to validate the HMF algorithm by comparing the deformed channels computed from the model with that derived from ultrasound imaging. In this case the eight corners of the phantom were used as the set of source points, and their new positions after applying a compression force as the target points. The computation took 0.4 s to complete in the same computer. The results are shown in Fig. 8.

In Fig. 8(a), the virtual phantom model and its host mesh are shown. The virtual phantom consists of a tetrahedra mesh and two channels. Profiles of the channels were shown via a cross-section of the phantom, which are compared with ultrasound images of Fig. 8(b). After superimposing the cross-section of the virtual model with the ultrasound image it was found that the profiles of deformed channel agreed with that shown in the ultrasonic images (Fig. 8c).

4 Discussion

Accurate prediction of tumour locations inside an organ intra-operationally is crucial for surgeons and interventional radiologists to achieve an optimal operational outcome. Breathing-induced organ displacement and deformation have been well studied, and algorithms to overcome the problem have also been described in literatures (see references [1, 2, 3, 4]). In general, it is not trivial to account for the breathing effects in an image registration process because organ displacement and deformation occur in all directions. Adding to this challenge is the fact that only part of the torso or organ surface are visible, thus to register the whole torso/organ model with limited intra-operational data becomes questionable. The key issue here is to seek a solution which not only yields a prediction for respiration-induced deformation accurate enough, but also has a light computational cost and a sufficient robustness.

It is in this context that we proposed the HMF algorithm, which is essentially an optimisation process yielding the best fitting between two set of data, that of the source and of the target points. The workflow of the algorithm is illustrated in Fig. 7, where the 5 steps in the diagram are described as follows:
  1. 1.

    Choose landmark points in a slave mesh as land mark points;

  2. 2.

    Compute elemental coordinates of the landmark points in the host mesh. Also note the corresponding target data points have the same elemental coordinates in the host mesh;

  3. 3.

    Generate a transformation matrix by minimisation of the projection from the source and target points;

  4. 4.

    Use the transformation matrix to drive the deformation of the host mesh;

  5. 5.

    The deformation of the host mesh in turn drive the deformation of the slave mesh.

Fig. 7.

Workflow of the HFM algorithm.

The advantage of this algorithm is that imaging warping comes with a minimal cost as it is implicitly done once the fitting algorithm is completed. Also the computation based on a small set of landmarks is fast (within one second), which is consistent with the surgical timeframe. The fast computation is achieved because solving elasticity equations is not required in the HMF algorithm. Therefore, even though fast finite element methods have been proposed (e.g., in [10]), applying these methods to nonlinear elasticity problems is non-trivial. Compared to the B-Spline interpolation model described in [2], where the displacement of one point is determined by an interpolation polynomial of its adjacent points, the HMF algorithm functions in a global scale rather than a local scale.

There are some limitations pertaining to the current method. For example, it is cumbersome to construct a slave mesh that encloses the target organ of interest. For instance, the liver organ was not distinguished from the torso mesh thus the hepatic motion secondary to respiration could not be effectively monitored. Since hepatic motion is a significant obstacle to precise needle placement [1], and also different abdominal organs have slightly different displacements [3], a new meshing strategy remains as our future work.
Fig. 8.

Validation experiments for the HMF algorithm were done in a gelatine phantom: (a) A virtual model for the phantom (slave mesh) is contained within a cube (host mesh) so that the deformation of the phantom is driven by the deformation of the cube. Also visible are the two channels whose profiles are compared with ultrasound images; (b) Ultrasonic imaging reveals a cross-section of the phantom before and after a compression force was applied; and (c) Deformation of the channel yielded by the host-mesh algorithm agrees with the ultrasonic data.

5 Conclusion

In this paper we introduced a Host Mesh Fitting algorithm that uses fiducial markers to drive the deformation of a host mesh which in turn transforms a slave mesh. The algorithm is able to morph CT images between different respiration states with minimum cost, therefore could be useful to help surgeons to better place ablation probes.


  1. 1.
    Clifford, M.A., Banovac, F., Levy, E., Cleary, K.: Assessment of hepatic motion secondary to respiration for computer assisted interventions. Comput. Aided Surg. 7, 291–299 (2002)CrossRefGoogle Scholar
  2. 2.
    Schreibmann, E., Chen, G.T.Y., Xing, L.: Image interpolation in 4D CT using a BSpline deformable registration model. Int. J. Radiat. Oncol. Biol. Phys. 64, 1537–1550 (2006)CrossRefGoogle Scholar
  3. 3.
    Brandner, E.D., Wu, A., Chen, H., Heron, D., Kalnicki, S., Komanduri, K., Gerszten, K., Burton, S., Ahmed, I., Shou, Z.: Abdominal organ motion measured using 4D CT. Int. J. Radiat. Oncol. Biol. Phys. 65, 554–560 (2006)CrossRefGoogle Scholar
  4. 4.
    Atoui, H., Miguet, S., Sarrut, D.: A fast morphing-based interpolation for medical images: application to conformal radiotherapy. Image Anal. Stere. 25, 95–103 (2011)CrossRefGoogle Scholar
  5. 5.
    Fernandez, J.W., Mithraratne, P., Thrupp, S.F., Tawhai, M.H., Hunter, P.J.: Anatomically based geometric modelling of the musculo-skeletal system and other organs. Biomech. Model. Mechanobiol. 2, 139–155 (2004)CrossRefGoogle Scholar
  6. 6.
    Ho, H., Bartlett, A., Hunter, P.: Geometric modelling of patient-specific hepatic structures using cubic hermite elements. In: Yoshida, H., Sakas, G., Linguraru, M.G. (eds.) ABD-MICCAI 2011. LNCS, vol. 7029, pp. 264–271. Springer, Heidelberg (2012). doi:10.1007/978-3-642-28557-8_33 CrossRefGoogle Scholar
  7. 7.
    Haelterman, R., Degroote, J., Van Heule, D., Vierendeels, J.: The quasi-newton least squares method: a new and fast secant method analyzed for linear systems. SIAM J. Numer. Anal. 47, 2347–2368 (2009)MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Kolr, P., Neuwirth, J., Sanda, J., Suchnek, V., Svat, Z., Volejnk, J., Pivec, M.: Analysis of diaphragm movement during tidal breathing and during its activation while breath holding using MRI synchronized with spirometry. Physiol. Res. 58, 383–392 (2009)Google Scholar
  9. 9.
    Farrer, A.I., Oden, H., de Bever, J., Coats, B., Parker, D.L., Payne, A., Christensen, D.A.: Characterization and evaluation of tissue-mimicking gelatin phantoms for use with MRgFUS. J. Ther. Ultrasound 3 (2015). doi:10.1186/s40349-015-0030-y
  10. 10.
    Bro-Nielsen, M.: Finite element modeling in surgery simulation. Proc. IEEE 86, 490–503 (1998)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Haobo Yu
    • 1
  • Harvey Ho
    • 1
  • Adam Bartlett
    • 2
  • Peter Hunter
    • 1
  1. 1.Auckland Bioengineering InstituteThe University of AucklandAucklandNew Zealand
  2. 2.Department of SurgeryThe University of AucklandAucklandNew Zealand

Personalised recommendations