A machine learning pipeline for internal anatomical landmark embedding based on a patient surface model

  • Xia ZhongEmail author
  • Norbert Strobel
  • Annette Birkhold
  • Markus Kowarschik
  • Rebecca Fahrig
  • Andreas Maier
Original Article



With the recent introduction of fully assisting scanner technologies by Siemens Healthineers (Erlangen, Germany), a patient surface model was introduced to the diagnostic imaging device market. Such a patient representation can be used to automate and accelerate the clinical imaging workflow, manage patient dose, and provide navigation assistance for computed tomography diagnostic imaging. In addition to diagnostic imaging, a patient surface model has also tremendous potential to simplify interventional imaging. For example, if the anatomy of a patient was known, a robotic angiography system could be automatically positioned such that the organ of interest is positioned in the system’s iso-center offering a good and flexible view on the underlying patient anatomy quickly and without any additional X-ray dose.


To enable such functionality in a clinical context with sufficiently high accuracy, we present an extension of our previous patient surface model by adding internal anatomical landmarks associated with certain (main) bones of the human skeleton, in particular the spine. We also investigate different approaches to positioning of these landmarks employing CT datasets with annotated internal landmarks as training data. The general pipeline of our proposed method comprises the following steps: First, we train an active shape model using an existing avatar database and segmented CT surfaces. This stage also includes a gravity correction procedure, which accounts for shape changes due to the fact that the avatar models were obtained in standing position, while the CT data were acquired with patients in supine position. Second, we match the gravity-corrected avatar patient surface models to surfaces segmented from the CT datasets. In the last step, we derive the spatial relationships between the patient surface model and internal anatomical landmarks.


We trained and evaluated our method using cross-validation using 20 datasets, each containing 50 internal landmarks. We further compared the performance of four different generalized linear models’ setups to describe the positioning of the internal landmarks relative to the patient surface. The best mean estimation error over all the landmarks was achieved using lasso regression with a mean error of \(12.19 \pm 6.98\ \hbox {mm}\).


Considering that interventional X-ray imaging systems can have detectors covering an area of about \(200\ \hbox {mm} \times 266\ \hbox {mm}\) (\(20\ \hbox {cm} \times 27\ \hbox {cm}\)) at iso-center, this accuracy is sufficient to facilitate automatic positioning of the X-ray system.


Patient modeling Anatomical landmark Statistical shape model Interventional X-ray Imaging 



We gratefully acknowledge the support of Siemens Healthineers, Forchheim, Germany. We thank Siemens Corporate Technology for providing the avatar database. Note that the concepts and information presented in this paper are based on research, and they are not commercially available.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.

Human and animal participants

This article does not contain any studies with human participants performed by any of the authors.


  1. 1.
    Allen B, Curless B, Popović Z (2003) The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans Graph 22:587–594CrossRefGoogle Scholar
  2. 2.
    Anguelov D, Koller D, Pang HC, Srinivasan P, Thrun S (2004) Recovering articulated object models from 3D range data. In: Uncertainty in artificial intelligence, pp 18–26Google Scholar
  3. 3.
    Au OKC, Tai CL, Chu HK, Cohen-Or D, Lee TY (2008) Skeleton extraction by mesh contraction. In: ACM transactions on graphics (TOG), vol 27. ACM, p 44Google Scholar
  4. 4.
    Bauer S, Wasza J, Haase S, Marosi N, Hornegger J (2011) Multi-modal surface registration for markerless initial patient setup in radiation therapy using Microsoft’s Kinect sensor. In: Proceedings of IEEE International Conference on Computer Vision, pp 1175–1181Google Scholar
  5. 5.
    Bednarek DR, Barbarits J, Rana VK, Nagaraja SP, Josan MS, Rudin S (2011) Verification of the performance accuracy of a real-time skin-dose tracking system for interventional fluoroscopic procedures. In: Medical imaging 2011: physics of medical imaging, vol 7961. International Society for Optics and Photonics, p 796127Google Scholar
  6. 6.
    Bork F, Barmaki R, Eck U, Yu K, Sandor C, Navab N (2017) Empirical study of non-reversing magic mirrors for augmented reality anatomy learning. In: 2017 IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, pp 169–176Google Scholar
  7. 7.
    Darom T, Keller Y (2012) Scale-invariant features for 3-D mesh models. IEEE Trans Image Process 21:2758–2769CrossRefGoogle Scholar
  8. 8.
    Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395CrossRefGoogle Scholar
  9. 9.
    Fletcher R (2013) Practical methods of optimization. Wiley, New YorkGoogle Scholar
  10. 10.
    Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 12(1):55–67CrossRefGoogle Scholar
  11. 11.
    Johnson PB, Borrego D, Balter S, Johnson K, Siragusa D, Bolch WE (2011) Skin dose mapping for fluoroscopically guided interventions. Med Phys 38(10):5490–5499CrossRefGoogle Scholar
  12. 12.
    Rausch J, Maier A, Fahrig R, Choi JH, Hinshaw W, Schebesch F, Haase S, Wasza J, Hornegger J, Riess C (2016) Kinect-based correction of overexposure artifacts in knee imaging with C-Arm CT systems. J Biomed Imaging 2016:1CrossRefGoogle Scholar
  13. 13.
    Robinette K.M, Daanen H, Paquet E (1999) The CAESAR project: a 3-D surface anthropometry survey. In: Second international conference on 3-D digital imaging and modeling, 1999. Proceedings. IEEE, pp 380–386Google Scholar
  14. 14.
    Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRefGoogle Scholar
  15. 15.
    Sahillioğlu Y, Yemez Y (2012) Minimum-distortion isometric shape correspondence using EM algorithm. IEEE Trans Pattern Anal Mach Intell 34(11):2203–2215CrossRefGoogle Scholar
  16. 16.
    Singh V, Chang Y.j, Ma K, Wels M, Soza G, Chen T (2014) Estimating a patient surface model for optimizing the medical scanning workflow. In: Medical image computing and computer-assisted intervention, pp 472–479Google Scholar
  17. 17.
    Taubmann O, Wasza J, Forman C, Fischer P, Wetzl J, Maier A, Hornegger J (2014) Prediction of respiration-induced internal 3-D deformation fields from dense external 3-D surface motion. In: 28th transaction on computer assisted radiology and surgery (CARS), pp 33–34Google Scholar
  18. 18.
    Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Ser B (Methodol) 58:267–288Google Scholar
  19. 19.
    Jimenez-del Toro O, Müller H, Krenn M, Gruenberg K, Taha AA, Winterstein M, Eggel I, Foncubierta-Rodríguez A, Goksel O, Jakab A (2016) Cloud-based evaluation of anatomical structure segmentation and landmark detection algorithms: visceral anatomy benchmarks. IEEE Trans Med Imaging 35(11):2459–2475CrossRefGoogle Scholar
  20. 20.
    Wasza J, Fischer P, Leutheuser H, Oefner T, Bert C, Maier A, Hornegger J (2016) Real-time respiratory motion analysis using 4-D shape priors. IEEE Trans Biomed Eng 63(3):485–495CrossRefGoogle Scholar
  21. 21.
    Zhong X, Strobel N, Sanders J, Kowarschik M, Fahrig R, Maier A (2017) Generation of personalized computational phantoms using only patient metadata. In: IEEE (ed) 2017 IEEE nuclear science symposium and medical imaging conference record (NSS/MIC)Google Scholar
  22. 22.
    Zou H, Hastie T (2005) Regularization and variable selection via the elastic net. J R Stat Soc Ser B (Stat Methodol) 67(2):301–320CrossRefGoogle Scholar

Copyright information

© CARS 2018

Authors and Affiliations

  • Xia Zhong
    • 1
    Email author
  • Norbert Strobel
    • 2
  • Annette Birkhold
    • 3
  • Markus Kowarschik
    • 3
  • Rebecca Fahrig
    • 3
  • Andreas Maier
    • 1
  1. 1.Pattern Recognition Laboratory, Department of Computer ScienceFriedrich-Alexander-Universität Erlangen-NürnbergErlangenGermany
  2. 2.Fakultät für ElektrotechnikHochschule für angewandte Wissenschaften Würzburg-SchweinfurtSchweinfurtGermany
  3. 3.Siemens Healthineers, Advanced TherapiesForchheimGermany

Personalised recommendations