Skip to main content
SpringerLink
Log in

4D segmentation algorithm with application to 3D+time image segmentation

  • Original Paper
  • Published:
Japan Journal of Industrial and Applied Mathematics Aims and scope Submit manuscript

Abstract

In this paper, we introduce and study a novel segmentation method for 4D images based on surface evolution governed by a nonlinear partial differential equation, the generalized subjective surface equation. The new method uses 4D digital image information and information from a thresholded 4D image in a local neighborhood. Thus, the 4D image segmentation is accomplished by defining the edge detector function’s input as the weighted sum of the norm of gradients of presmoothed 4D image and norm of presmoothed thresholded 4D image in a local neighborhood. Additionally, we design and study a numerical method based on the finite volume approach for solving the new model. The reduced diamond cell approach is used for approximating the gradient of the solution. We use a semi-implicit finite volume scheme for the numerical discretization and show that our numerical scheme is unconditionally stable. The new 4D method was tested on artificial data and applied to real data representing 3D+time microscopy images of cell nuclei within the zebrafish pectoral fin and hind-brain. In a real application, processing 3D+time microscopy images amounts to solving a linear system with several billion unknowns and requires over 1000 GB of memory; thus, it may not be possible to process these images on a serial machine without parallel implementation utilizing the MPI. Consequently, we develop and present in the paper OpenMP and MPI parallel implementation of designed algorithms. Finally, we include cell tracking results to show how our new method serves as a basis for finding trajectories of cells during embryogenesis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19

Similar content being viewed by others

References

  1. Amdahl, G. M.: Validity of the single-processor approach to achieving large scale computing capabilities. AFIPS Conf. Proc. (30):483–485

  2. Aoyama, Y., Nakano, J.: RS/6000 SP: Practical MPI Programming. IBM www.redbooks.ibm.com (1999)

  3. Briggs, K.: Diagonally Dominant Matrix. From MathWorld–A Wolfram Web Resource, created by Eric W. Weisstein. https://mathworld.wolfram.com/DiagonallyDominantMatrix.html

  4. Bungert, L., Aizinger, V., Fried, M.: A Discontinuous Galerkin Method for the Subjective Surfaces Problem. J Math Imaging Vis 58, 147–161 (2017)

    Article  MATH  Google Scholar 

  5. Corsaro, S., Mikula, K., Sarti, A., Sgallari, F.: Semi-implicit co-volume method in 3D image segmentation. SIAM J. Sci. Comput. 28(6), 2248–2265 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  6. Dervieux, A., Thomasset, F.: A finite element method for the simulation of a Rayleigh–Taylor instability. In: Rautmann R. (eds) Approximation Methods for Navier–Stokes Problems. Lecture Notes in Mathematics, Springer, Berlin vol 771 (1980)

  7. Dervieux, A., Thomasset, F.: Multifluid incompressible flows by a finite element method. In: Reynolds W. C., MacCormack R. W. (eds) Seventh International Conference on Numerical Methods in Fluid Dynamics. Lecture Notes in Physics, Springer, Berlin, vol. 141 (1981)

  8. Digabel, H., Lantuejoul, C.: Iterative Algorithms. In: Proceedings of 2nd European Symposium on Quantitative Analysis of Micro-structures in Material Science, pp. 85–99, Stuttgart (1978). Riederer

  9. Dyballa, S., Savy, T., Germann, P., Mikula, K., Remesikova, M., Špir, R., Zecca, A., Peyriéras, N., Pujades, C.: Distribution of neurosensory progenitor pools during inner ear morphogenesis unveiled by cell lineage reconstruction. eLife 6, e22268 (2017). https://doi.org/10.7554/eLife.22268

  10. Evans, L.C., Spruck, J.: Motion of level sets by mean curvature I. J. Differ. Geom. 33, 635–681 (1991). https://doi.org/10.4310/jdg/1214446559

    Article  MathSciNet  MATH  Google Scholar 

  11. Falcao, A.X., Udupa, J.K.: Segmentation of 3D Objects using Live Wire. Proc. SPIE Med. Imaging 3034(1), 228–239 (1997)

    Article  Google Scholar 

  12. Falcao, A. X., Udupa, J. K., Samarasekera, S., Sharma, S., Hirsch, B. E., de Alencar Lofufo, R.: User-steered image segmentation paradigms: Live-wire and live-lane. Graph. Models Image Process. 60(4), 223–260 (1998)

  13. Falcao, A.X., Jayaram, K., Udupa, J.K., Miyazawa, F.K.: An ultra-fast usersteered image segmentation paradigm: live-wire-on-the fly. Proc. SPIE Med. Imaging 3661, 184–191 (1999)

    Article  Google Scholar 

  14. Faure, E., Savy, T., Rizzi, B., Melani, C., Drblíková, O., Fabregès, D., Špir, R., Hammons, M., Čunderlík, R., Recher, G., Lombardot, B., Duloquin, L., Colin, I., Kollár, J., Desnoulez, S., Affaticati, P., Maury, B., Boyreau, A., Nief, J., Calvat, P., Vernier, P., Frain, M., Lutfalla, G., Kergosien, Y., Suret, P., Remešíková, M., Doursat, R., Sarti, A., Mikula, K., Peyriéras, N., Bourgine, P.: An algorithmic workflow for the automated processing of 3D+time microscopy imaging of developing organisms and reconstruction of their cell lineage. Nat. Commun. 7, 8674 (2016)

    Article  Google Scholar 

  15. Handlovičová, A., Mikula, K., Sgallari, F.: Semi-implicit complementary volume scheme for solving level set like equations in image processing and curve evolution. Numer. Math. 93, 675–695 (2003). https://doi.org/10.1007/s002110100374

    Article  MathSciNet  MATH  Google Scholar 

  16. Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. Int J Comput Vis 1(4), 321–331 (1988)

  17. Koenderink, J.: The structure of images. Biol. Cybern. 50, 363–370 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  18. Mikula, K., Peyriéras, N., Remešíková, M., Stašová, O.: Segmentation of 3D cell membrane images by PDE methods and its applications. Comput. Biol. Med. 41(6), 326–339 (2011)

  19. Mikula, K., Remešíková, M.: Finite volume schemes for the generalized subjective surface equation in image segmentation. Kybernetika 45(4), 646–656 (2009)

    MathSciNet  MATH  Google Scholar 

  20. Mikula, K., Peyriéras, N., Remešíková, M., Smíšek, M.: 4D Numerical Schemes for Cell Image Segmentation and Tracking. In: Fořt J., Fürst J., Halama J., Herbin R., Hubert F. (eds) Finite Volumes for Complex Applications VI Problems & Perspectives. Springer Proc. Math. 4(2011). https://doi.org/10.1007/978-3-642-20671-9_73

  21. Mikula, K., Špir, R., Smíšek, M., Faure, E., Peyriéras, N.: Nonlinear PDE based numerical methods for cell tracking in zebrafish embryogenesis. Appl. Numer. Math. 95, 250–266 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  22. Mikula, K., Špir, R., Peyriéras, N.: Parallelization and validation of algorithms for zebrafish cell lineage tree reconstruction from big 4D image data. Acta Polytech Hungar. 14(5), 65–84 (2017)

    Google Scholar 

  23. Mikula, K., Sarti, A.: Parallel co-volume subjective surface method for 3D medical image segmentation. Parametric and geometric deformable models: an application in Biomaterials and Medical Imagery, Volume-II, Springer Publishers, (Eds. Jasjit S. Suri and Aly Farag), ISBN: 0-387-31204-8, pp. 123–160 (2007)

  24. Mikula, K., Sarti, A., Sgallari, F.: Co-volume level set method in subjective surface based medical image segmentation. In: Handbook of medical image analysis: segmentation and registration models (J. Suri et al., Eds.), Springer, New York, pp. 583–626 (2005)

  25. Mittal, S.: A Study of Successive Over-relaxation (SOR) method parallelization over modern HPC languages. Int. J. High Perform. Comput. Netw. 7(4), 292–298 (2015)

  26. Osher, S., Sethian, J. A.: Fronts propagating with curvature-dependent speed: Algorithms based on Hamilton-Jacobi formulations. J. Comput. Phys. 79(1), 12–49 (1988)

  27. Otsu, N.A.: A threshold selection method from gray-level histograms. IEEE Trans. Syst. Man Cybern. 9(1), 62–66 (1979)

    Article  Google Scholar 

  28. Park, S.: Tracking cells in 3D + time image sequences – written part of dissertation exam. Slovak University of Technology in Bratislava, Faculty of Civil Engineering (2020)

  29. Park, S., Sipka, T., Kriva, Z., Ambroz, M., Kollár, M., Kósa, B., Nguyen-Chi, M., Lutfalla, G., Mikula, K.: Macrophage image segmentation by thresholding and subjective surface method. Tatra Mt. Math. Publ. 75, 103–120 (2020)

    MathSciNet  MATH  Google Scholar 

  30. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639 (1990). https://doi.org/10.1109/34.56205

    Article  Google Scholar 

  31. Preim, B., Bartz, D.: Image analysis for medical visualization. Vis. Med. 83–133 (2007). https://doi.org/10.1016/b978-012370596-9/50007-9

  32. Sarti, A., Malladi, R., Sethian, J. A.: Subjective surfaces: a geometric model for boundary completion. Int. J. Comput. Vis. 46(3), 201–221 (2002)

  33. Sarti, A., Malladi, R., Sethian, J. A.: Subjective surfaces: a method for completing missing boundaries. PNAS 12(97), 6258–6263 (2000)

  34. Sethian, J.A.: Level set methods and fast marching methods: evolving interfaces in computational geometry, fluid mechanics, computer vision, and material science. Cambridge University Press, New York (1999)

    MATH  Google Scholar 

  35. Skeel, R. D.: Numerical Linear Algebra. Purdue University, pp. 1–41 (2006). https://www.cs.purdue.edu/homes/skeel/CS515/1.pdf

  36. Uba, M.O., Mikula, K., Krivá, Z., Nguyen, H., Savy, T., Kardash, E., Peyriéras, N.: 3D cell image segmentation by modified subjective surface method. Tatra Mt. Math. Publ. 75, 147–162 (2020). https://doi.org/10.2478/tmmp-2020-0010

    Article  MathSciNet  Google Scholar 

  37. Uba, M. O., Mikula, K., Krivá, Z., Nguyen, H., Savy, T., Kardash, E., Peyriéras, N.: Application of modified subjective surface method to 3d cell membrane image segmentation. ALGORITMY 2020, Conference on Scientific Computing. Podbanske, Slovakia, Proceedings of contributed papers, 51–60 (2020)

  38. Vincent, L., Soille, P.: Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans. Pattern Anal. Mach. Intell. 13(6), 583–598 (1991)

    Article  Google Scholar 

  39. Witkin, P.: Scale-space filtering. In: Proc. 8th Int. Joint Conf. Art. Intel l., (Karlsruhe, West Germany), pp. 1019–1022 (1983)

  40. Zanella, C., Campana, M., Rizzi, B., Melani, C., Sanguinetti, G., Bourgine, P., Mikula, K., Peyrieras, N., Sarti, A.: Cells segmentation from 3-D confocal images of early zebrafish embryogenesis. IEEE Trans. Image Process. 19(3), 770–781 (2010)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No. 721537, and the projects APVV-19-0460 and VEGA 1/0436/20.

Author information

Authors and Affiliations

  1. Department of Mathematics and Descriptive Geometry, Slovak University of Technology in Bratislava, Radlinského 11, 810 05, Bratislava, Slovakia

    Markjoe Olunna Uba, Karol Mikula & Seol Ah Park

  2. Department of Mathematics, University of Nigeria, Nsukka - Onitsha Rd, 410 001, Nsukka, Nigeria

    Markjoe Olunna Uba

Authors
  1. Markjoe Olunna Uba
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Karol Mikula
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seol Ah Park
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Markjoe Olunna Uba.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Uba, M.O., Mikula, K. & Park, S.A. 4D segmentation algorithm with application to 3D+time image segmentation. Japan J. Indust. Appl. Math. 40, 109–139 (2023). https://doi.org/10.1007/s13160-022-00519-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13160-022-00519-w

Keywords

Mathematics Subject Classification

Search

Navigation