Advertisement

iModel: Interactive Co-segmentation for Object of Interest 3D Modeling

  • Adarsh Kowdle
  • Dhruv Batra
  • Wen-Chao Chen
  • Tsuhan Chen
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6554)

Abstract

We present an interactive system to create 3D models of objects of interest in their natural cluttered environments.

A typical setting for 3D modeling of an object of interest involves capturing images from multiple views in a multi-camera studio with a mono-color screen or structured lighting. This is a tedious process and cannot be applied to a variety of objects. Moreover, general scene reconstruction algorithms fail to focus on the object of interest to the user. In this paper, we use successful ideas from the object cut-out literature, and develop an interactive-cosegmentation-based algorithm that uses scribbles from the user indicating foreground (object to be modeled) and background (clutter) to extract silhouettes of the object of interest from multiple views. Using these silhouettes, and the camera parameters obtained from structure-from-motion, in conjunction with a shape-from-silhouette algorithm we generate a texture-mapped 3D model of the object of interest.

Keywords

Multiple View Camera Parameter Control Setup Photo Collection Interactive Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download to read the full conference paper text

References

  1. 1.
    Levoy, M., Pulli, K., Curless, B., Rusinkiewicz, S., Koller, D., Pereira, L., Ginzton, M., Anderson, S., Davis, J., Ginsberg, J., Shade, J., Fulk, D.: The digital michelangelo project: 3d scanning of large statues. In: Siggraph, pp. 131–144 (2000)Google Scholar
  2. 2.
    Szeliski, R.: Rapid octree construction from image sequences. CVGIP: Image Understanding 58, 23–32 (1993)CrossRefGoogle Scholar
  3. 3.
    Fang, Y.H., Chou, H.L., Chen, Z.: 3d shape recovery of complex objects from multiple silhouette images. Pattern Recogn. Lett. 24, 1279–1293 (2003)CrossRefzbMATHGoogle Scholar
  4. 4.
    Chen, Z., Chou, H.L., Chen, W.C.: A performance controllable octree construction method. In: ICPR, pp. 1–4 (2008)Google Scholar
  5. 5.
    Forbes, K., Nicolls, F., de Jager, G., Voigt, A.: Shape-from-Silhouette with Two Mirrors and an Uncalibrated Camera. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 165–178. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  6. 6.
    Rother, C., Kolmogorov, V., Blake, A.: “Grabcut”: interactive foreground extraction using iterated graph cuts. In: SIGGRAPH (2004)Google Scholar
  7. 7.
    Hochbaum, D.S., Singh, V.: An efficient algorithm for co-segmentation. In: ICCV (2009)Google Scholar
  8. 8.
    Mukherjee, L., Singh, V., Dyer, C.R.: Half-integrality based algorithms for cosegmentation of images. In: CVPR (2009)Google Scholar
  9. 9.
    Batra, D., Kowdle, A., Parikh, D., Luo, J., Chen, T.: icoseg: Interactive co-segmentation with intelligent scribble guidance. In: CVPR (2010)Google Scholar
  10. 10.
    Snavely, N., Seitz, S., Szeliski, R.: Photo tourism: Exploring photo collections in 3d. In: SIGGRAPH, pp. 835–846 (2006)Google Scholar
  11. 11.
    Franco, J.S., Boyer, E.: Exact polyhedral visual hulls. In: BMVC, vol. 1, pp. 329–338 (2003)Google Scholar
  12. 12.
    Starck, J., Hilton, A.: Surface capture for performance-based animation. IEEE Computer Graphics and Applications 27, 21–31 (2007)CrossRefGoogle Scholar
  13. 13.
    Vlasic, D., Baran, I., Matusik, W., Popović, J.: Articulated mesh animation from multi-view silhouettes. In: SIGGRAPH, pp. 1–9. ACM (2008)Google Scholar
  14. 14.
    Curless, B., Levoy, M.: A volumetric method for building complex models from range images. In: SIGGRAPH, pp. 303–312. ACM (1996)Google Scholar
  15. 15.
    Chen, Y., Medioni, G.: Object modelling by registration of multiple range images. Image Vision Comput. 10, 145–155 (1992)CrossRefGoogle Scholar
  16. 16.
    Fitzgibbon, A.W., Cross, G., Zisserman, A.: Automatic 3D Model Construction for Turn-Table Sequences. In: Koch, R., Van Gool, L. (eds.) SMILE 1998. LNCS, vol. 1506, pp. 155–170. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  17. 17.
    Zhang, L., Curless, B., Seitz, S.M.: Rapid shape acquisition using color structured light and multi-pass dynamic programming. In: 3DPVT, vol. 24 (2002)Google Scholar
  18. 18.
    Zhang, L., Curless, B., Seitz, S.M.: Spacetime stereo: Shape recovery for dynamic scenes. In: CVPR, vol. 2, p. 367 (2003)Google Scholar
  19. 19.
    Yezzi, A., Soatto, S.: Stereoscopic segmentation. IJCV 53, 31–43 (2003)CrossRefGoogle Scholar
  20. 20.
    Lee, W., Woo, W., Boyer, E.: Identifying Foreground from Multiple Images. In: Yagi, Y., Kang, S.B., Kweon, I.S., Zha, H. (eds.) ACCV 2007, Part II. LNCS, vol. 4844, pp. 580–589. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  21. 21.
    Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from internet photo collections. IJCV 80, 189–210 (2008)CrossRefGoogle Scholar
  22. 22.
    Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: CVPR, vol. 1, pp. 519–528 (2006)Google Scholar
  23. 23.
    Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. PAMI 32, 1362–1376 (2010)CrossRefGoogle Scholar
  24. 24.
    Vergauwen, M., Van Gool, L.: Web-based 3d reconstruction service. Mach. Vision Appl. 17, 411–426 (2006)CrossRefGoogle Scholar
  25. 25.
    Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S.M.: Multi-view stereo for community photo collections. In: ICCV, pp. 265–270 (2007)Google Scholar
  26. 26.
    Furukawa, Y., Curless, B., Seitz, S.M., Szeliski, R.: Towards internet-scale multi-view stereo. In: CVPR (2010)Google Scholar
  27. 27.
    Campbell, N., Vogiatzis, G., Hernndez, C., Cipolla, R.: Automatic 3d object segmentation in multiple views using volumetric graph-cuts. In: BMVC (2007)Google Scholar
  28. 28.
    Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary and region segmentation of objects in n-d images. In: ICCV (2001)Google Scholar
  29. 29.
    Li, Y., Sun, J., Tang, C.K., Shum, H.Y.: Lazy snapping. In: SIGGRAPH (2004)Google Scholar
  30. 30.
    Sormann, M., Zach, C., Bauer, J., Karner, K., Bischof, H.: Automatic Foreground Propagation in Image Sequences for 3D Reconstruction. In: Kropatsch, W.G., Sablatnig, R., Hanbury, A. (eds.) DAGM 2005. LNCS, vol. 3663, pp. 93–100. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  31. 31.
    Hengel, A., Dick, A.R., Thormhlen, T., Ward, B., Torr, P.H.S.: Videotrace: rapid interactive scene modelling from video. ACM Trans. Graph. 26, 86 (2007)CrossRefGoogle Scholar
  32. 32.
    Sinha, S., Steedly, D., Szeliski, R., Agrawala, M., Pollefeys, M.: Interactive 3d architectural modeling from unordered photo collections. ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia 2008) (2008)Google Scholar
  33. 33.
    Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. PAMI 24, 603–619 (2002)CrossRefGoogle Scholar
  34. 34.
    Bagon, S.: Matlab wrapper for graph cut (2006)Google Scholar
  35. 35.
    Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. PAMI 26, 1124–1137 (2004)CrossRefGoogle Scholar
  36. 36.
    Boykov, Y., Veksler, O., Zabih, R.: Efficient approximate energy minimization via graph cuts. PAMI 20, 1222–1239 (2001)CrossRefGoogle Scholar
  37. 37.
    Kolmogorov, V., Zabih, R.: What energy functions can be minimized via graph cuts? PAMI 26, 147–159 (2004)CrossRefGoogle Scholar
  38. 38.
    Fabbri, R., Kimia, B.B.: 3D curve sketch: Flexible curve-based stereo reconstruction and calibration. In: CVPR (2010)Google Scholar
  39. 39.
    Chen, W.C., Chou, H.L., Chen, Z.: A quality controllable multi-view object reconstruction method for 3d imaging systems. JVCIR 21, 427–441 (2010)Google Scholar
  40. 40.
    Wong, K.Y.K., Cipolla, R.: Reconstruction of sculpture from its profiles with unknown camera positions. IEEE Transactions on Image Processing 13, 381–389 (2004)CrossRefGoogle Scholar
  41. 41.
    Wong, K.Y.K., Cipolla, R.: Structure and motion from silhouettes. In: ICCV (2001)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2012

Authors and Affiliations

  • Adarsh Kowdle
    • 1
  • Dhruv Batra
    • 2
  • Wen-Chao Chen
    • 3
  • Tsuhan Chen
    • 1
  1. 1.Cornell UniversityIthacaUSA
  2. 2.Carnegie Mellon UniversityPittsburghUSA
  3. 3.Industrial Technology Research InstituteTaiwan

Personalised recommendations

Cite paper