Simulation of Ground-Truth Validation Data Via Physically- and Statistically-Based Warps

  • Ghassan Hamarneh
  • Preet Jassi
  • Lisa Tang
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5241)


The problem of scarcity of ground-truth expert delineations of medical image data is a serious one that impedes the training and validation of medical image analysis techniques. We develop an algorithm for the automatic generation of large databases of annotated images from a single reference dataset. We provide a web-based interface through which the users can upload a reference data set (an image and its corresponding segmentation and landmark points), provide custom setting of parameters, and, following server-side computations, generate and download an arbitrary number of novel ground-truth data, including segmentations, displacement vector fields, intensity non-uniformity maps, and point correspondences. To produce realistic simulated data, we use variational (statistically-based) and vibrational (physically-based) spatial deformations, nonlinear radiometric warps mimicking imaging non-homogeneity, and additive random noise with different underlying distributions. We outline the algorithmic details, present sample results, and provide the web address to readers for immediate evaluation and usage.


validation segmentation deformation simulation vibration variation non-uniformity 

Supplementary material

978-3-540-85988-8_55_MOESM1_ESM.pdf (954 kb)
Supplementary Material (954 KB)


  1. 1.
    Alexander, D., Pierpaoli, C., Basser, P., Gee, J.: Spatial transformations of diffusion tensor magnetic resonance images. IEEE TMI 20(11), 1131–1139 (2001)Google Scholar
  2. 2.
    Chou, Y., Skrinjar, O.: Ground truth data for validation of nonrigid image registration Algorithms. In: ISBI, pp. 716–719 (2004)Google Scholar
  3. 3.
    Christensen, et al.: Introduction to the Non-Rigid Image Registration Evaluation Project (NIREP). In: Biomedical Image Registration Workshop, pp. 128–135 (2006)Google Scholar
  4. 4.
    Cocosco, C., Kollokian, V., Kwan, R., Evans, A.: BrainWeb: Online Interface to a 3D MRI Simulated Brain Database. NeuroImage 5(4), part 2/4, S425 (1997)Google Scholar
  5. 5.
    Cootes, T., Edwards, Taylor, C.: Active Appearance Models. PAMI 23(6), 681–685 (2001)Google Scholar
  6. 6.
    Cootes, T., Taylor, C., Cooper, D., Grahamet, J.: Active Shape Models - Their Training and Application. Computer Vision and Image Understanding 61(1), 38–59 (1995)CrossRefGoogle Scholar
  7. 7.
    Cootes, T., Taylor, C.: Combining point distribution models with shape models based on finite element analysis. Image and Vision Computing 13(5), 403–409 (1995)CrossRefGoogle Scholar
  8. 8.
    Davies, R., Twining, C., Cootes, T., Waterton, J., Taylor, C.: A minimum description length approach to statistical shape Modeling. IEEE TMI 21(5), 525–537 (2002)Google Scholar
  9. 9.
    Dice, L.: Measures of the Amount of Ecologic Association Between Species. Ecology 26(3), 297–302 (1945)CrossRefGoogle Scholar
  10. 10.
    Everingham, M., Muller, H., Thomas, B.: Evaluating image segmentation algorithms using monotonic hulls in fitness/cost space. BMVC, 363–372 (2001)Google Scholar
  11. 11.
    Everingham, M., Muller, H., Thomas, B.: Evaluating image segmentation algorithms using the Pareto front. ECCV (IV), 34–48 (2002)Google Scholar
  12. 12.
    Fitzpatrick, et al.: Visual assessment of the accuracy of retrospective registration of MR and CT images of the brain. IEEE TMI 17, 571–585 (1998)Google Scholar
  13. 13.
    Gerig, G., Jomier, M., Chakos, M.: VALMET: A new validation tool for assessing and improving 3D object segmentation. In: Niessen, W.J., Viergever, M.A. (eds.) MICCAI 2001. LNCS, vol. 2208, pp. 516–523. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  14. 14.
    Hellier, et al.: Retrospective Evaluation of Inter-subject Brain Registration. IEEE TMI 22(9), 1120–1130 (2003)Google Scholar
  15. 15.
    Internet Brain Segmentation Repository,
  16. 16.
    Jaccard, P.: Étude comparative de la distribution florale dans une portion des Alpes et des Jura. Bulletin de la Société Vaudoise des Sciences Naturelles 37, 547–579 (1901)Google Scholar
  17. 17.
    Karlsson, J., Ericsson, A.: A ground truth correspondence measure for benchmarking. In: ICPR, pp. 568–573 (2006)Google Scholar
  18. 18.
    Kwan, R., Evans, A., Pike, B.: MRI Simulation-Based Evaluation of Image-Processing and Classification Methods. IEEE TMI 18(11), 1085–1097 (1999)Google Scholar
  19. 19.
    Lehman, T., Gonner, C., Spitzer, K.: Survey: Interpolation Methods in Medical Image Processing. IEEE TMI 18(11), 1049–1075 (1999)Google Scholar
  20. 20.
    Maintz, J., Viergever, M.: A survey of medical image registration. MIA 2(1), 1–36 (1998)Google Scholar
  21. 21.
    Maurer, C., Fitzpatrick, J., Wang, M., Galloway, R., Maciunas, R., Allen, G.: Registration of head volume images using implantable fiducial markers. IEEE TMI 16(4), 447–462 (1997)Google Scholar
  22. 22.
    Pawluczyk, O., Yaffe, M.: Field nonuniformity correction for quantitative analysis of digitized mammograms. Medical Physics 28(4), 438–444 (2001)CrossRefGoogle Scholar
  23. 23.
    Pennec, X., Thirion, J.: A Framework for Uncertainty and Validation of 3-D Registration Methods based on Points and Frames. IJCV 25(3), 203–229 (1997)CrossRefGoogle Scholar
  24. 24.
    Reilhac, et al.: PET-SORTEO: validation and development of database of Simulated PET volumes. IEEE Transactions on Nuclear Science 52(5), part 1, 1321–1328 (2005)Google Scholar
  25. 25.
    Rosenberger, C.: Adaptive evaluation of image segmentation results. In: ICPR, pp. 399–402 (2006)Google Scholar
  26. 26.
    Schestowitz, R., Twining, C., Petrovic, V., Cootes, T., Crum, B., Taylor, C.: Non-Rigid Registration Assessment Without Ground Truth. In: MIUA, vol. 2, pp. 151–155 (2006)Google Scholar
  27. 27.
    Sled, J., Pike, B.: Understanding Intensity Non-uniformity in MRI. In: Wells, W.M., Colchester, A.C.F., Delp, S.L. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 614–622. Springer, Heidelberg (1998)Google Scholar
  28. 28.
    Styner, M., Rajamani, K., Nolte, L., Zsemlye, G., Székely, G., Taylor, C., Davies, R.: Evaluation of 3D Correspondence Methods for Model Building. In: Taylor, C.J., Noble, J.A. (eds.) IPMI 2003. LNCS, vol. 2732, pp. 63–75. Springer, Heidelberg (2003)Google Scholar
  29. 29.
    Twining, C., Cootes, T., Marsland, S., Petrovic, V., Schestowitz, R., Taylor, C.: Information-Theoretic Unification of Groupwise Non-Rigid Registration and Model Building. In: MIUA, pp. 226–230 (2006)Google Scholar
  30. 30.
    Unnikrishnan, R., Pantofaru, C., Hebert, M.: A Measure for Objective Evaluation of Image Segmentation Algorithms. In: CVPR Workshop on Empirical Methods in Computer Vision, p. 34 (2005)Google Scholar
  31. 31.
    Warfield, S., Zho, K., Wells, W.: Simultaneous Truth and Performance Level Estimation (STAPLE): An Algorithm for the Validation of Image Segmentation. IEEE TMI 23(7), 903–921 (2004)Google Scholar
  32. 32.
    West, et al.: Comparison and evaluation of retrospective inter-modality brain image registration techniques. J. Computer Assisted Tomography 21(4), 554–566 (1997)Google Scholar
  33. 33.
    Zhang, Y.: A review of recent evaluation methods for image segmentation. In: ISSPA, pp. 148–151 (2001)Google Scholar
  34. 34.
    Zhang, H., Fritts, J., Goldman, S.: An entropy-based objective evaluation methods for image segmentation. In: SPIE, vol. 5307, pp. 38–49 (2003)Google Scholar
  35. 35.
    Zhang, H., Cholleti, S., Goldman, S.: Meta-Evaluation of Image Segmentation Using Machine Learning. In: CVPR, pp. 1138–1145 (2006)Google Scholar
  36. 36.
    Zhang, Y.: A survey on Evaluation Methods for Image Segmentation. Pattern Recognition 29(8), 1335–1346 (1996)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Ghassan Hamarneh
    • 1
  • Preet Jassi
    • 1
  • Lisa Tang
    • 1
  1. 1.Medical Image Analysis Lab.Simon Fraser UniversityBurnabyCanada

Personalised recommendations