Advertisement

Machine Vision and Applications

, Volume 25, Issue 6, pp 1615–1629 | Cite as

Graph-cut based interactive segmentation of 3D materials-science images

  • Jarrell Waggoner
  • Youjie Zhou
  • Jeff Simmons
  • Marc De Graef
  • Song WangEmail author
Original Paper

Abstract

Segmenting materials’ images is a laborious and time-consuming process, and automatic image segmentation algorithms usually contain imperfections and errors. Interactive segmentation is a growing topic in the areas of image processing and computer vision, which seeks to find a balance between fully automatic methods and fully-manual segmentation processes. By allowing minimal and simplistic interaction from the user in an otherwise automatic algorithm, interactive segmentation is able to simultaneously reduce the time taken to segment an image while achieving better segmentation results. Given the specialized structure of materials’ images and level of segmentation quality required, we show an interactive segmentation framework for materials’ images that has three key contributions: (1) a multi-labeling approach that can handle a large number of structures while still quickly and conveniently allowing manual addition and removal of segments in real-time, (2) multiple extensions to the interactive tools which increase the simplicity of the interaction, and (3) a web interface for using the interactive tools in a client/server architecture. We show a full formulation of each of these contributions and example results from their application.

Keywords

Image segmentation Materials volume segmentation Segmentation propagation Interactive segmentation Graph-cut approaches 

Notes

Acknowledgments

This work was supported in part by AFOSR FA9550-11-1-0327 and NSF-1017199. A preliminary version of this work has been published in a conference proceedings [59].

Supplementary material

Supplementary material 1 (avi 9465 KB)

References

  1. 1.
    Batra, D., Kowdle, A., Parikh, D., Luo, J., Chen, T.: iCoseg: Interactive co-segmentation with intelligent scribble guidance. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3169–3176 (2010)Google Scholar
  2. 2.
    Birkbeck, N., Cobzas, D., Jagersand, M., Murtha, A., Kesztyues, T.: An interactive graph cut method for brain tumor segmentation. In: Workshop on Applications of Computer Vision (WACV), pp. 1–7 (2009)Google Scholar
  3. 3.
    Boykov, Y., Jolly, M.: Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In: IEEE International Conference on Computer Vision, vol. 1, pp. 105–112. IEEE Press, New York (2001)Google Scholar
  4. 4.
    Boykov, Y., Jolly, M.P.: Interactive organ segmentation using graph cuts. In: Delp, S., DiGoia, A., Jaramaz, B. (eds.) Medical Image Computing and Computer-Assisted Intervention MICCAI 2000. Lecture Notes in Computer Science, vol. 1935, pp. 147–175. Springer, Berlin (2000)Google Scholar
  5. 5.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)CrossRefGoogle Scholar
  6. 6.
    Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)CrossRefGoogle Scholar
  7. 7.
    Bradski, G.: The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)Google Scholar
  8. 8.
    Canny, J.: A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8(6), 679–698 (1986)CrossRefGoogle Scholar
  9. 9.
    Cates, J.E., Lefohn, A., Whitaker, R.T.: Gist: An interactive, gpu-based level-set segmentation. Med. Image Anal. 8(3), 217–231 (2004). http://graphics.cs.ucdavis.edu/lefohn/work/rls/tumorSeg/
  10. 10.
    Chuang, H., Huffman, L., Comer, M., Simmons, J., Pollak, I.: An automated segmentation for nickel-based superalloy. In: IEEE International Conference on Image Processing, pp. 2280–2283 (2008)Google Scholar
  11. 11.
    Comer, M., Bouman, C., De Graef, M., Simmons, J.: Bayesian methods for image segmentation. J. Miner. Metals Mater. Soc. 63, 55–57 (2011)CrossRefGoogle Scholar
  12. 12.
    Comer, M., Delp, E.: Parameter estimation and segmentation of noisy or textured images using the EM algorithm and MPM estimation. In: IEEE International Conference on Image Processing, vol. 2, pp. 650–654. IEEE Press, New York (1994)Google Scholar
  13. 13.
    Comer, M., Delp, E.: The EM/MPM algorithm for segmentation of textured images: analysis and further experimental results. IEEE Trans. Image Process. 9(10), 1731–1744 (2000)CrossRefzbMATHGoogle Scholar
  14. 14.
    Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995)zbMATHGoogle Scholar
  15. 15.
    Django Software Foundation: Django (version 1.5). http://djangoproject.com
  16. 16.
    Fragkiadaki, K., Zhang, W., Shi, J., Bernardis, E.: Structural-flow trajectories for unravelling tubular structure bundles. In: Medical Image Computing and Computer-Assisted Intervention, vol. 3, pp. 631–638 (2012)Google Scholar
  17. 17.
    Gonzalez, R.C., Woods, R.E.: Digital Image Processing, 3rd edn. Prentice Hall, New York (2008)Google Scholar
  18. 18.
    Heckel, F., Konrad, O., Hahn, H.K., Peitgen, H.O.: Interactive 3D medical image segmentation with energy-minimizing implicit functions. Comput. Graph. 35(2), 275–287 (2011)CrossRefGoogle Scholar
  19. 19.
    Huffman, L., Simmons, J., Pollak, I.: Segmentation of digital microscopy data for the analysis of defect structures in materials using nonlinear diffusion. In: C. Bouman, E. Miller, I. Pollak (eds.) Computational Imaging VI, Proceedings of SPIE (2008)Google Scholar
  20. 20.
    Huffman, L.M., Simmons, J.P., De Graef, M., Pollak, I.: Shape priors for map segmentation of alloy micrographs using graph cuts. In: IEEE Workshop on Statistical, Signal Processing, pp. 28–30 (2011)Google Scholar
  21. 21.
    Ibrahim, I.A., Mohamed, F.A., Lavernia, E.J.: Particulate reinforced metal matrix composites: a review. J. Mater. Sci. 26, 1137–1156 (1991)CrossRefGoogle Scholar
  22. 22.
    Jacinto, H., Kchichan, R., Desvignes, M., Prost, R., Valette, S.: A web interface for 3D visualization and interactive segmentation of medical images. In: 17th International Conference on 3D Web Technology (Web 3D 2012), pp. 51–58 (2012)Google Scholar
  23. 23.
    Jackson, M., Groeber, M.: DREAM3D (2012). http://dream3d.bluequartz.net
  24. 24.
    Jones, E., Oliphant, T., Peterson, P., et al.: SciPy: Open source scientific tools for Python (2001). http://www.scipy.org/
  25. 25.
    Kang, Y., Engelke, K., Kalender, W.A.: Interactive 3D editing tools for image segmentation. Med. Image Anal. 8(1), 35–46 (2004)CrossRefGoogle Scholar
  26. 26.
    Kiefer, W.: Intelligent scissoring for interactive segmentation of 3D meshes. Master’s thesis, Princeton University (2004)Google Scholar
  27. 27.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 147–159 (2004)CrossRefGoogle Scholar
  28. 28.
    Kuang, Z., Schnieders, D., Zhou, H., Wong, K.Y., Yu, Y., Peng, B.: Learning image-specific parameters for interactive segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 590–597 (2012)Google Scholar
  29. 29.
    Li, Q., Ni, X., Liu, G.: Ceramic image processing using the second curvelet transform and watershed algorithm. In: IEEE International Conference on Robotics and Biomimetics, pp. 2037–2042 (2007)Google Scholar
  30. 30.
    Marroquin, J., Mitter, S., Poggio, T.: Probabilistic solution of ill-posed problems in computational vision. J. Am. Stat. Assoc. 76–89 (1987)Google Scholar
  31. 31.
    Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE International Conference on Computer Vision, vol. 2, pp. 416–423 (2001)Google Scholar
  32. 32.
    Mortensen, E.N., Barrett, W.A.: Intelligent scissors for image composition. In: Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques. SIGGRAPH ’95, pp. 191–198. ACM, New York (1995)Google Scholar
  33. 33.
    Moschidis, E., Graham, J.: Propagating interactive segmentation of a single 3D example to similar images: an evaluation study using MR images of the prostate. In: IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1472–1475 (2011)Google Scholar
  34. 34.
    Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)zbMATHMathSciNetGoogle Scholar
  35. 35.
    Pfister, S.S., Betizeau, M., Dehay, C., Douglas, R.J.: INTERSEG: Interactive 3D segmentation (2012). http://n.ethz.ch/student/sabpfist/interseg.htm
  36. 36.
    Python Software Foundation: Python language reference. http://www.python.org
  37. 37.
    Reed, R.: The Superalloys: Fundamentals and Applications. Cambridge University Press, Cambridge (2006)CrossRefGoogle Scholar
  38. 38.
    Rollett, A., Gottstein, G., Shvindlerman, L., Molodov, D.: Grain boundary mobility: a brief review. Z. Metallkunde 95, 226–229 (2004)CrossRefGoogle Scholar
  39. 39.
    Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (Proceedings of SIGGRAPH) 23(3), 309–314 (2004) Google Scholar
  40. 40.
    Rowenhorst, D., Lewis, A., Spanos, G.: Three-dimensional analysis of grain topology and interface curvature in a \(\beta \)-titanium alloy. Acta. Mater. 58, 5511–5519 (2010)CrossRefGoogle Scholar
  41. 41.
    Santner, J., Pock, T., Bischof, H.: Interactive multi-label segmentation. In: Asian Conference on Computer Vision, pp. 397–410 (2011)Google Scholar
  42. 42.
    Schneider, C., Rasband, W., Eliceiri, K.: NIH image to ImageJ: 25 years of image analysis. Nat. Methods 9, 671–675 (2012)Google Scholar
  43. 43.
    Settles, B.: Active learning literature survey. Computer Sciences Technical Report 1648, University of Wisconsin-Madison (2009)Google Scholar
  44. 44.
    Shalev-Shwartz, S.: Online Learning: Theory, Algorithms, and Applications. Ph.D. thesis, The Hebrew University of Jerusalem (2007)Google Scholar
  45. 45.
    Shapiro, L.G., Stockman, G.C.: Computer Vision. Prentice Hall, Upper Saddle River (2001)Google Scholar
  46. 46.
    Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)CrossRefGoogle Scholar
  47. 47.
    Simmons, J., Bartha, B., De Graef, M., Comer, M.: Development of bayesian segmentation techniques for automated segmentation of titanium alloy images. Microsc. Microanal. 14(S2), 602–603 (2008)CrossRefGoogle Scholar
  48. 48.
    Simmons, J.P., Chuang, P., Comer, M., Spowart, J.E., Uchic, M.D., De Graef, M.: Application and further development of advanced image processing algorithms for automated analysis of serial section image data. Model. Simul. Mater. Sci. Eng. 17(2), 025,002 (2009)CrossRefGoogle Scholar
  49. 49.
    Straehle, C., Koethe, U., Knott, G., Briggman, K., Denk, W., Hamprecht, F.: Seeded watershed cut uncertainty estimators for guided interactive segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 765–772 (2012)Google Scholar
  50. 50.
    Straehle, C.N., Köthe, U., Knott, G., Hamprecht, F.A.: Carving: scalable interactive segmentation of neural volume electron microscopy images. In: Medical Image Computing and Computer-Assisted Intervention, pp. 653–660 (2011)Google Scholar
  51. 51.
    Swiler, T.P., Holm, E.A.: Diffusion in polycrystalline microstructures. In: Annual Meeting of the American Ceramic Society (1995)Google Scholar
  52. 52.
    Tan, J., Saltzman, W.: Biomaterials with hierarchically defined micro and nanoscale structure. Biomaterials 25(17), 3593–3601 (2004)CrossRefGoogle Scholar
  53. 53.
    Top, A., Hamarneh, G., Abugharbieh, R.: Active learning for interactive 3D image segmentation. In: Medical Image Computing and Computer-Assisted Intervention vol. 6893, pp. 603–610 (2011)Google Scholar
  54. 54.
    Unger, M., Pock, T., Trobin, W., Cremers, D., Bischof, H.: TVSeg–interactive total variation based image segmentation. In: British Machine Vision Conference 2008, pp. 40.1–40.10 (2008)Google Scholar
  55. 55.
    Veksler, O.: Efficient graph-based energy minimization methods in computer vision. Ph.D. thesis, Cornell University, Ithaca (1999)Google Scholar
  56. 56.
    Veksler, O., Delong, A.: GCO (2011). http://vision.csd.uwo.ca/code/
  57. 57.
    Vezhnevets, V., Konouchine, V.: Grow-Cut–interactive multi-label N-D image segmentation. In: Graphicon, pp. 150–156 (2005)Google Scholar
  58. 58.
    Waggoner, J., Zhou, Y., Simmons, J., De Graef, M., Wang, S.: 3D materials image segmentation by 2D propagation: a graph-cut approach considering homomorphism. IEEE Trans. Image Process. 22, 5282–5293 (2013)CrossRefGoogle Scholar
  59. 59.
    Waggoner, J., Zhou, Y., Simmons, J., Salem, A., De Graef, M., Wang, S.: Interactive grain image segmentation using graph cut algorithms. In: Proceedings of SPIE (Computational Imaging XI), vol. 8657. Burlingame (2013)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Jarrell Waggoner
    • 1
  • Youjie Zhou
    • 1
  • Jeff Simmons
    • 2
  • Marc De Graef
    • 3
  • Song Wang
    • 1
    Email author
  1. 1.University of South CarolinaColumbiaUSA
  2. 2.Materials and Manufacturing DirectorateAir Force Research LabsDaytonUSA
  3. 3.Department of Materials Science and EngineeringCarnegie Mellon UniversityPittsburghUSA

Personalised recommendations