The Visual Computer

, Volume 32, Issue 5, pp 625–640 | Cite as

A robust hybrid image-based modeling system

  • Hoang Minh NguyenEmail author
  • Burkhard Wünsche
  • Patrice Delmas
  • Christof Lutteroth
  • Eugene Zhang
Original Article


This paper presents a new robust image-based modeling system for creating high-quality 3D models of complex objects from a sequence of unconstrained photographs. The images can be acquired by a video camera or hand-held digital camera without the need of camera calibration. In contrast to previous methods, we integrate correspondence-based and silhouette-based approaches, which significantly enhances the reconstruction of objects with few visual features (e.g., uni-colored objects) and improves surface smoothness. Our solution uses a mesh segmentation and charting approach in order to create a low-distortion mesh parameterization suitable for objects of arbitrary genus. A high-quality texture is produced by first parameterizing the reconstructed objects using a segmentation and charting approach, projecting suitable sections of input images onto the model, and combining them using a graph-cut technique. Holes in the texture due to surface patches without projecting input images are filled using a novel exemplar-based inpainting method which exploits appearance space attributes to improve patch search, and blends patches using Poisson-guided interpolation. We analyzed the effect of different algorithm parameters, and compared our system with a laser scanning-based reconstruction and existing commercial systems. Our results indicate that our system is robust, superior to other image-based modeling techniques, and can achieve a reconstruction quality visually not discernible from that of a laser scanner.


3D shape recovery Texture acquisition Surface reconstruction Surface parameterization  Texture inpainting 


  1. 1.
    Nguyen, M.H., Wunsche, B., Delmas, P., Lutteroth, C.: A hybrid image-based modelling algorithm. In: Proceedings of 36th Australasian Computer Science Conference (ACSC 2013) (2013)Google Scholar
  2. 2.
    Nguyen, H.M., Wunsche, B., Delmas, P., Lutteroth, C.: 3D models from the black box: investigating the current state of image-based modeling. In: WSCG 2012 Communication Proceedings, pp. 249–258 (2012)Google Scholar
  3. 3.
    Hernandez, C., Vogiatzis, G., Cipolla, R.: Multi-view photometric stereo. IEEE Trans. Pattern Recognit. Mach. Intell. 30, 548–554 (2008)CrossRefGoogle Scholar
  4. 4.
    Franco, J.-S., Lapierre, M., Boyer, E.: Visual shapes of silhouette sets. In: 3D Data Processing, Visualization and Transmission, pp. 397–404 (2006)Google Scholar
  5. 5.
    Matusik, W., Buehler, C., Raskar, R., Gortler, S., McMillan, L.: Image-based visual hulls. In: Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 369–374 (2000)Google Scholar
  6. 6.
    Nguyen, M.H., Wunsche, B., Delmas, P., Lutteroth, C.: Realistic 3D scene reconstruction from unconstrained and uncalibrated images. In: Proceedings of GRAPP 2011, Algarve, Portugal, vol. 31, pp. 67–75 (2011)Google Scholar
  7. 7.
    Baumgart, B.G.: Geometric modeling for computer vision. Doctoral Dissertation, Stanford University (1974)Google Scholar
  8. 8.
    Grauman, K., Shakhnarovich, G., Darrell, T.: A Bayesian approach to image-based visual hull reconstruction. In: IEEE International Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 187–194 (2003)Google Scholar
  9. 9.
    Cheung, K., Baker, S., Kanade, T.: Shape-from-silhouette across time part 1: theory and algorithms. Int. J. Comput. Vis. 62(1), 221–247 (2005)CrossRefGoogle Scholar
  10. 10.
    Cheung, K., Baker, S., Kanade, T.: Shape-from-silhouette across time part 2: applications to human modeling and markerless motion tracking. Int. J. Comput. Vis. 63(1), 225–245 (2005)CrossRefGoogle Scholar
  11. 11.
    Franco, J.S., Boyer, E.: Exact polyhedral visual hulls. In: British Machine Vision Conference, pp. 329–338 (2003)Google Scholar
  12. 12.
    Debevec, P.E., Taylor, C.J., Malik, J.: Modeling and rendering architecture from photographs: a hybrid geometry and image-based approach. ACM Trans Graph. pp. 11–20 (1996)Google Scholar
  13. 13.
    Quan, L., Tan, P., Zeng, G., Yuan, L., Wang, J., Kang, S.B.: Image-based plant modeling. ACM Trans. Graph. 25(3), 599–604 (2006)CrossRefGoogle Scholar
  14. 14.
    Brown, M., Lowe, D.G.: Unsupervised 3D object recognition and reconstruction in unordered datasets. In: Fifth International Conference on 3D Digital Imaging and Modeling, pp. 56–63 (2005)Google Scholar
  15. 15.
    Snavely, N., Seitz, S., Szeliski, R.: Photo tourism: exploring photo collections in 3D. ACM Trans. Graph. 25(3), 835–846 (2006)CrossRefGoogle Scholar
  16. 16.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRefGoogle Scholar
  17. 17.
    Lowe, D.G.: Object recognition from local scale-invariant features. In: The Proceedings of the Seventh IEEE International Conference on Computer Vision, vol. 2, pp. 1150–1157 (1999)Google Scholar
  18. 18.
    Nguyen, H.M.: Accelerated 3D Content Creation using Stereo from Motion. Master Thesis. The University of Auckland, New Zealand (2012)Google Scholar
  19. 19.
    Remondino, F., El-Hakim, S.: Image-based 3D modelling: a review. Photogramm. Rec. 21, 269–291 (2006)CrossRefGoogle Scholar
  20. 20.
    Garai, G., Chaudhuri, B.B.: A split and merge procedure for polygonal border detection of dot pattern. Image Vis. Comput. 17, 75–82 (1999)CrossRefGoogle Scholar
  21. 21.
    Amenta, N., Choi, S., Kolluri, R.K.: The power crust. Int. J. Comput. Geom. Theory Appl. 19, 127–153 (2000)MathSciNetCrossRefzbMATHGoogle Scholar
  22. 22.
    Edelsbrunner, H., Mucke, E.P.: Three-dimensional alpha shapes. ACM Trans. Graph. 13, 43–72 (1994)CrossRefzbMATHGoogle Scholar
  23. 23.
    Bernardini, F., Mittleman, J., Rushmeier, H., Silva, C., Taubin, G.: The ball-pivoting algorithm for surface reconstruction. IEEE Trans. Vis. Comput. Graph. 5(4), 349–359 (1999)CrossRefGoogle Scholar
  24. 24.
    Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Proceedings of the Fourth Eurographics Symposium on Geometry Processing, pp. 61–70 (2006)Google Scholar
  25. 25.
    Zhang, E., Mischaikow, K., Turk, G.: Feature-based surface parameterization and texture mapping. ACM Trans. Graph. 24(1), 1–27 (2005)CrossRefGoogle Scholar
  26. 26.
    Reeb, Georges: Sur les points singuliers dune forme de pfaff completement integrable ou diune fonction numerique [on the (singular points of a completely integrable pfaff form or of a numerical function)]. Comptes Randus Acad. Sci. Paris 222, 847–849 (1946)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Hilaga, M., Shinagawa, Y., Komura, T., Kunii, T.L.: Topology matching for fully automatic similarity estimation of 3D shapes. In: Computer Graphics Proceedings, Annual Conference Series (SIGGRAPH 2001), pp. 203–212 (2001)Google Scholar
  28. 28.
    Eck, M., DeRose, T., Duchamp, T., Hoppey, H., Lounsberyz, M., Stuetzle, W.: Multi-resolution analysis of arbitrary meshes. In: Computer Graphics Proceedings, Computer Graphics Proceedings, Annual Conference Series (SIGGRAPH 1995), pp. 173–182 (1995)Google Scholar
  29. 29.
    Floater, S.M.: Parametrization and smooth approximation of surface triangulations. Comput. Aided Geom. Des. 14(3), 231–250 (1997)MathSciNetCrossRefzbMATHGoogle Scholar
  30. 30.
    Sander, P.V., Gortler, S.J., Snyder, J., Hoppe, H.: Signal-specialized parameterization, In: Proceedings of the 13th Eurographics Workshop on Rendering, pp. 87–100 (2002)Google Scholar
  31. 31.
    Kwata, V., Schodl, A., Essaa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22(3), 277–286 (2003)CrossRefGoogle Scholar
  32. 32.
    Clark, X.B., Finlay, J., Wilson, A., Milburn, K., Nguyen, M.H., Lutteroth, C., Wunsche, B.C.: An investigation into graphcut parameter optimisation for image-fusion applications. In: Proceedings of Image and Vision Computing New Zealand (IVCNZ 2012), pp. 480–485 (2012)Google Scholar
  33. 33.
    Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceeding SIGGRAPH ’00 Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques, pp. 417–424 (2000)Google Scholar
  34. 34.
    Telea, A.: An image inpainting technique based on the fast marching method. J. Graph. Tools 9(1), 23–34 (2004)CrossRefGoogle Scholar
  35. 35.
    Perez, P., Gangnet, M., Blake, A.: Poisson image editing. J. ACM Trans. Graph. 22(3), 313–318 (2003)CrossRefGoogle Scholar
  36. 36.
    Criminisi, A., Perez, P., Toyama, K.: Object removal by exemplar-based inpainting. IEEE Trans. Image Process. 19(9), 1200–1212 (2004)CrossRefGoogle Scholar
  37. 37.
    Harrison, P.: A non-hierarchical procedure for re-synthesis of complex texture. In: Proceedings of International Conference on Graphics, Visualisation and Computer Vision, pp. 190–197 (2001)Google Scholar
  38. 38.
    Manke, F., Wunsche, B.: Fast spatially controllable 2D/3D texture synthesis and morphing for multiple input textures. In: Proceedings of the 4th International Conference on Computer Graphics Theory and Applications (GRAPP 2009), pp. 5–12 (2009)Google Scholar
  39. 39.
    Lefebvre, S., Hoppe, H.: Appearance-space texture synthesis. In: ACM SIGGRAPH, pp. 541–548 (2006)Google Scholar
  40. 40.
    Nguyen, H.M., Wunsche, B., Delmas, P., Lutteroth, C.: Parameter optimisation for texture reconstruction. In: Proceedings of Image and Vision Computing New Zealand (IVCNZ 2013), pp. 226–230 (2013)Google Scholar
  41. 41.
    Stanford Bunny. Accessed 1st May 2014
  42. 42.
    Rakhmanov, E.A., Saff, E.B., Zhou, Y.M.: Minimal discrete energy on the sphere. Math. Res. Lett. 1(6), 647–662 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Rusinkiewicz, S., Levoy, M.: Efficient Variants of the ICP Algorithm. In: Proceeding of Third International Conference on 3D Digital Imaging and Modeling (3DIM), pp. 145–152 (2001)Google Scholar
  44. 44.
    Zambanini, S., Kampel, M.: A local image descriptor robust to illumination changes. In: Proceedings of the 18th Scandinavian Conference on Image Analysis, pp. 11–21 (2013)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  • Hoang Minh Nguyen
    • 1
    Email author
  • Burkhard Wünsche
    • 1
  • Patrice Delmas
    • 1
  • Christof Lutteroth
    • 1
  • Eugene Zhang
    • 2
  1. 1.University of AucklandAucklandNew Zealand
  2. 2.Oregon State UniversityCorvallisUSA

Personalised recommendations