Skip to main content
SpringerLink
Log in

Techniques for Seamless Color Registration and Mapping on Dense 3D Models

  • Chapter
  • First Online:
Sensing the Past

Part of the book series: Geotechnologies and the Environment ((GEOTECH,volume 16))

Abstract

Today’s most widely used 3D digitization approach is a combination of active geometric sensing, mainly using laser scanning, with active or passive color sensing, mostly using digital photography. Producing a seamless colored object, starting from a geometric representation and a set of photographs, is a data fusion problem requiring effective solutions for image-to-geometry registration, and color mapping and blending. This chapter provides a brief survey of the state-of-the-art solutions, ranging from manual approaches to fully scalable automated methods.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Similar content being viewed by others

References

  • Agus M, Gobbetti E, Villanueva AJ, Mura C, Pajarola R (2014) SOAR: Stochastic optimization for affine global point set registration. In: Proceedings of the 19th international workshop on vision, modeling and visualization (VMV), Oct 2014, Darmstadt, pp 103–110

    Google Scholar 

  • Aiger D, Mitra NJ, Cohen-Or D (2008) 4-points congruent sets for robust pairwise surface registration. ACM Trans Graph (TOG) 27:85

    Article  Google Scholar 

  • Allène C, Philippe Pons J, Keriven R (2008) Seamless image-based texture atlases using multi-band blending. In: Proceeding of the ICPR. IEEE, pp 1–4

    Google Scholar 

  • Arikan M, Preiner R, Scheiblauer C, Jeschke S, Wimmer M (2014) Large-scale point-cloud visualization through localized textured surface reconstruction. IEEE Trans Vis Comput Graph 20(9):1280–1292

    Article  Google Scholar 

  • Banno A, Ikeuchi K (2010) Omnidirectional texturing based on robust 3D registration through Euclidean reconstruction from two spherical images. Comput Vis Image Underst 114(4):491–499

    Article  Google Scholar 

  • Baumberg A (2002) Blending images for texturing 3D models. In: Proceeding of the BMVC, Citeseer, vol 3, p 5

    Google Scholar 

  • Berkiten S, Fan X, Rusinkiewicz S (2014) Merge2-3D: combining multiple normal maps with 3D surfaces. In: 2nd international conference on 3D vision, Dec 2014, vol 1. IEEE, Los Alamitos/Washington/Tokyo, pp 440–447

    Chapter  Google Scholar 

  • Bettio F, Gobbetti E, Merella E, Pintus R (2013) Improving the digitization of shape and color of 3D artworks in a cluttered environment. In: Proceedings of the digital heritage, Oct 2013. IEEE, Piscataway, pp 23–30. Best Paper Award

    Google Scholar 

  • Bettio F, Villanueva AJ, Merella E, Marton F, Gobbetti E, Pintus R (2015) Mont’e Scan: effective shape and color digitization of cluttered 3D artworks. J Comput Cult Herit (JOCCH) 8(1):4:1–4:23

    Google Scholar 

  • Bernardini F, Martin IM, Rushmeier H (2001) High-quality texture reconstruction from multiple scans. IEEE Trans Vis Comput Graph 7(4):318–332

    Article  Google Scholar 

  • Bernardini F, Rushmeier H (2002) The 3D model acquisition pipeline. Comput Gr Forum 21:149–172

    Article  Google Scholar 

  • Borgeat L, Poirier G, Beraldin A, Godin G, Massicotte P, Picard M (2009) A framework for the registration of color images with 3D models. In: Proceedings of the 16th international conference on image processing (ICIP’09), IEEE Press, Piscataway, NJ, USA, pp 69–72

    Google Scholar 

  • Brunie L, Lavallée S, Szeliski R (1992) Using force fields derived from 3D distance maps for inferring the attitude of a 3D rigid object. In: Computer Vision ECCV’92. Springer, pp 670–675

    Google Scholar 

  • Callieri M, Cignoni P, Corsini M, Scopigno R (2008) Masked photo blending: mapping dense photographic data set on high-resolution sampled 3D models. Comput Graph 32(4):464–473

    Article  Google Scholar 

  • Callieri M, Cignoni P, Scopigno R (2002) Reconstructing textured meshes from multiple range RGB maps. In: VMV, Erlangen, pp 419–426

    Google Scholar 

  • Chuang M, Luo L, Brown BJ, Rusinkiewicz S, Kazhdan M (2009) Estimating the laplace-beltrami operator by restricting 3D functions. Comput Gr Forum 28:1475–1484. Wiley Online Library

    Google Scholar 

  • Cleju I, Saupe D (2007) Stochastic optimization of multiple texture registration using mutual information. Springer, Konstanz

    Book  Google Scholar 

  • Corsini M, Callieri M, Cignoni P (2008) Stereo light probe. Comput Gr Forum 27:291–300. Wiley Online Library

    Google Scholar 

  • Corsini M, Dellepiane M, Ganovelli F, Gherardi R, Fusiello A, Scopigno R (2013) Fully automatic registration of image sets on approximate geometry. Int J Comput Vis 102(1–3):91–111

    Article  Google Scholar 

  • Corsini M, Dellepiane M, Ponchio F, Scopigno R (2009) Image-to-geometry registration: a mutual information method exploiting illumination-related geometric properties. Comput Gr Forum 28:1755–1764. Wiley Online Library

    Google Scholar 

  • Debevec P, Yizhou Yu, Borshukov G (1998) Efficient view-dependent image-based rendering with projective texture-mapping. Springer, Berkeley

    Book  Google Scholar 

  • Dellepiane M, Benedetti L, Scopigno R (2010) Removing shadows for color projection using sun position estimation. In: Proceedings of the 11th international conference on virtual reality, archaeology and cultural heritage. Eurographics Association, pp 55–62

    Google Scholar 

  • Dellepiane M, Callieri M, Corsini M, Cignoni P, Scopigno R (2010) Improved color acquisition and mapping on 3D models via flash-based photography. J Comput Cult Herit (JOCCH) 2(4):9

    Google Scholar 

  • Dellepiane M, Callieri M, Ponchio F, Scopigno R (2008) Mapping highly detailed colour information on extremely dense 3D models: the case of David’s restoration. Comput Gr Forum 27:2178–2187. Wiley Online Library

    Google Scholar 

  • Dellepiane M, Marroquim R, Callieri M, Cignoni P, Scopigno R (2012) Flow-based local optimization for image-to-geometry projection. IEEE Trans Vis Comput Graph 18(3):463–474

    Article  Google Scholar 

  • Dellepiane M, Scopigno R (2013) Global refinement of image-to-geometry registration for color projection on 3D models. In: Digital heritage international congress. The Eurographics Association, pp 39–46

    Google Scholar 

  • Eisemann M, De Decker B, Magnor M, Bekaert P, De Aguiar E, Ahmed N, Theobalt C, Sellent A (2008) Floating textures. Comput Gr Forum 27:409–418. Wiley Online Library

    Google Scholar 

  • Franken T, Dellepiane M, Ganovelli F, Cignoni P, Montani C, Scopigno R (2005) Minimizing user intervention in registering 2D images to 3D models. Vis Commun 21(8–10):619–628

    Article  Google Scholar 

  • Frahm JM, Fite-Georgel P, Gallup D, Johnson T, Raguram R, Wu C, Jen Y-H, Dunn E, Clipp B, Lazebnik S et al (2010) Building Rome on a cloudless day. In: Computer vision–ECCV 2010. Springer, pp 368–381

    Google Scholar 

  • Fruh C, Zakhor A (2001) Fast 3D model generation in urban environments. In: Proceedings of the MFI. IEEE, pp 165–170

    Google Scholar 

  • Fruh C, Zakhor A (2003) Constructing 3D city models by merging aerial and ground views. Comput Gr Appl IEEE 23(6):52–61

    Article  Google Scholar 

  • Gal R, Wexler Y, Ofek E, Hoppe H, Cohen-Or D (2010) Seamless montage for texturing models. Comput Gr Forum 29:479–486. Wiley Online Library

    Google Scholar 

  • Goldluecke B, Cremers D (2009) Superresolution texture maps for multiview reconstruction. In: Proceedings of the ICCV. IEEE, pp 1677–1684

    Google Scholar 

  • Guo R, Dai Q, Hoiem D (2011) Single-image shadow detection and removal using paired regions. In: Proceedings of the CVPR. IEEE, pp 2033–2040

    Google Scholar 

  • Haber T, Fuchs C, Bekaer P, Seidel H-P, Goesele M, Lensch HP et al (2009) Relighting objects from image collections. In: Proceedings of the CVPR. IEEE, pp 627–634

    Google Scholar 

  • Hantak C, Lastra A (2006) Metrics and optimization techniques for registration of color to laser range scans. In: International symposium on 3D data processing, visualization, and transmission, third. IEEE, pp 551–558

    Google Scholar 

  • Ikeuchi K, Oishi T, Takamatsu J, Sagawa R, Nakazawa A, Kurazume R, Nishino K, Kamakura M, Okamoto Y (2007) The Great Buddha project: digitally archiving, restoring, and analyzing cultural heritage objects. Int J Comput Vis 75(1):189–208

    Article  Google Scholar 

  • Kaminsky RS, Snavely N, Seitz SM, Szeliski R (2009) Alignment of 3D point clouds to overhead images. In: Proceedings of the CVPR workshops. IEEE, pp 63–70

    Google Scholar 

  • Koutsoudis A, Vidmar B, Ioannakis G, Arnaoutoglou F, Pavlidis G, Chamzas C (2014) Multi-image 3D reconstruction data evaluation. J Cult Herit 15(1):73–79

    Article  Google Scholar 

  • Kim SJ, Lin HT, Lu Z, Suesstrunk S, Lin S, Brown MS (2012) A new in-camera imaging model for color computer vision and its application. IEEE Trans PAMI 34(12):2289–2302

    Article  Google Scholar 

  • Lensch H, Heidrich W, Seidel H-P (2000) Automated texture registration and stitching for real world models. In: Proceedings of the pacific graphics. IEEE, pp 317–452

    Google Scholar 

  • Lensch HPA, Heidrich W, Seidel H-P (2001) A silhouette-based algorithm for texture registration and stitching. Graph Model 63(4):245–262

    Article  Google Scholar 

  • Lempitsky V, Ivanov D (2007) Seamless Mosaicing of image-based texture maps. In: Proceedings of the CVPR. IEEE, pp 1–6

    Google Scholar 

  • Levoy M, Pulli K, Curless B, Rusinkiewicz S, Koller D, Pereira L, Ginzton M, Anderson S, Davis J, Ginsberg J et al (2000) The digital michelangelo project: 3D scanning of large statues. In: Proceedings of the SIGGRAPH. ACM, New York, pp 131–144

    Google Scholar 

  • Li Y, Low K-L (2009) Automatic registration of color images to 3D geometry. In: Proceedings of the CGI. CGS, pp 21–28

    Google Scholar 

  • Liu L, Stamos I (2005) Automatic 3D to 2D registration for the photorealistic rendering of urban scenes. In: Proceedings of the CVPR. IEEE, vol 2, pp 137–143

    Google Scholar 

  • Liu L, Stamos I, Gene Yu, Wolberg G, Zokai S (2006) Multiview geometry for texture mapping 2D images onto 3D range data. In: Proceedings of the CVPR. IEEE, vol 2, pp 2293–2300

    Google Scholar 

  • Lowe DG (1991) Fitting parameterized three-dimensional models to images. IEEE Trans Pattern Anal Mach Intell 13(5):441–450

    Article  Google Scholar 

  • Malzbender T, Gelb D, Wolters H (2001) Polynomial texture maps. In: Proceedings of the 28th annual conference on computer graphics and interactive techniques. ACM, pp 519–528

    Google Scholar 

  • Mastin A, Kepner J, Fisher J (2009) Automatic registration of LIDAR and optical images of urban scenes. In: Proceedings of the CVPR. IEEE, pp 2639–2646

    Google Scholar 

  • Matsushita K, Kaneko T (1999) Efficient and handy texture mapping on 3D surfaces. Comput Gr Forum, Blackwell Publishers, Oxford, UK, vol 18, pp 349–358

    Google Scholar 

  • Nehab D, Rusinkiewicz S, Davis J, Ramamoorthi R (2005) Efficiently combining positions and normals for precise 3D geometry. ACM Trans Graph (TOG) 24(3):536–543

    Article  Google Scholar 

  • Neugebauer PJ, Klein K (1999) Texturing 3D models of real world objects from multiple unregistered photographic views. Comput Gr Forum 18:245–256. Wiley Online Library

    Google Scholar 

  • Palma G, Callieri M, Dellepiane M, Scopigno R (2012) A statistical method for SVBRDF approximation from video sequences in general lighting conditions. Comput Gr Forum 31:1491–1500. Wiley Online Library

    Google Scholar 

  • Pintus R, Gobbetti E (2015) A fast and robust framework for semi-automatic and automatic registration of photographs to 3D geometry. J Comput Cult Herit (JOCCH) 7(4):23:1–23:23

    Google Scholar 

  • Pintus R, Gobbetti E, Callieri M (2011a) Fast low-memory seamless photo blending on massive point clouds using a streaming framework. J Comput Cult Herit (JOCCH) 4(2):6

    Google Scholar 

  • Pintus R, Gobbetti E, Callieri M (2011b) A streaming framework for seamless detailed photo blending on massive point clouds. In: Proceedings of the eurographics area papers. pp 25–32

    Google Scholar 

  • Pintus R, Gobbetti E, Combet R (2011c) Fast and robust semi-automatic registration of photographs to 3D geometry. In: The 12th international symposium on virtual reality, archaeology and cultural heritage Oct 2011. Eurographics Association, pp 9–16

    Google Scholar 

  • Pulli K, Abi-Rached H, Duchamp T, Shapiro LG, Stuetzle W (1998) Acquisition and visualization of colored 3D objects. In: Proceedings of the ICPR. IEEE, pp 11–15

    Google Scholar 

  • Remondino F (2011) Heritage recording and 3D modeling with photogrammetry and 3D scanning. Remote Sens 3(6):1104–1138

    Article  Google Scholar 

  • Rocchini C, Cignoni P, Montani C, Scopigno R (2002) Acquiring, stitching and blending diffuse appearance attributes on 3D models. Vis Comput 18(3):186–204

    Article  Google Scholar 

  • Sequeira V, Gonçalves GMJ (2002) 3D reality modelling: photo-realistic 3D models of real world scenes. In: Proceedings of the 3DPVT. IEEE, pp 776–783

    Google Scholar 

  • Sinha SN, Steedly D, Szeliski R, Agrawala M, Pollefeys M (2008) Interactive 3D architectural modeling from unordered photo collections. ACM Trans Gr (TOG) 27:159

    Google Scholar 

  • Stamos I, Alien PK (2001) Automatic registration of 2-D with 3-D imagery in urban environments. In: Proceedings of the ICCV. IEEE, vol 2, pp 731–736

    Google Scholar 

  • Stamos I, Liu L, Chen C, Wolberg G, Yu G, Zokai S (2008) Integrating automated range registration with multiview geometry for the photorealistic modeling of large-scale scenes. Int J Comput Vis 78(2–3):237–260

    Article  Google Scholar 

  • Viola P, Wells WM III (1997) Alignment by maximization of mutual information. Int J Comput Vis 24(2):137–154

    Article  Google Scholar 

  • Waechter M, Moehrle N, Goesele M (2014) Let there be color! large-scale texturing of 3D reconstructions. In: Computer vision–ECCV 2014. Springer, pp 836–850

    Google Scholar 

  • Walkowski F, Johnston RA, Price NB (2008) Texture mapping for the Fastscan hand-held laser scanner. In: Proceedings of the IVCNZ. IEEE, pp 1–6

    Google Scholar 

  • Williams N, Low K-L, Hantak C, Pollefeys M, Lastra A (2004) Automatic image alignment for 3D environment modeling. In: Proceedings of the computer graphics and image processing. IEEE, pp 388–395

    Google Scholar 

  • Wu C, Clipp B, Xiaowei Li, Frahm J-M, Pollefeys M (2008) 3D model matching with viewpoint-invariant patches (VIP). In: Proceedings of the CVPR. IEEE, pp 1–8

    Google Scholar 

  • Xu S-C, Ye X, Wu Y, Zhang S (2005) Highlight detection and removal based on chromaticity. In: Image analysis and recognition. Springer, pp 199–206

    Google Scholar 

  • Xu L, Li E, Li J, Chen Y, Zhang Y (2010) A general texture mapping framework for image-based 3D modeling. In: Proceedings of the international conference on image processing, Hong kong, pp 2713–2716

    Google Scholar 

  • Yang G, Becker J, Stewart CV (2007) Estimating the location of a camera with respect to a 3D model. In: Proceedings of the 3DIM. IEEE, pp 159–166

    Google Scholar 

  • Zhao W, Nister D, Hsu S (2005) Alignment of continuous video onto 3D point clouds. IEEE Trans Pattern Anal Mach Intell 27(8):1305–1318

    Article  Google Scholar 

  • Zheng H, Cleju I, Saupe D (2010) Highly-automatic mi based multiple 2D/3D image registration using self-initialized geodesic feature correspondences. In: Computer vision–ACCV 2009. Springer, pp 426–435

    Google Scholar 

  • Zhou Q-Y, Koltun V (2014) Color map optimization for 3D reconstruction with consumer depth cameras. ACM Trans Gr (TOG) 33(4):155

    Google Scholar 

Download references

Acknowledgements

This work is partially supported by the EU H2020 Program under the Scan4Reco project (665091) funded under H2020-EU.3.6.3. We also acknowledge the contribution of Sardinian Regional authorities for Visual Computing activities (projects VIGEC and HELIOS).

Author information

Authors and Affiliations

  1. CRS4, Visual Computing Group, Cagliari, Italy

    Ruggero Pintus & Enrico Gobbetti

  2. ISTI-CNR, Visual Computing Group, Pisa, Italy

    Marco Callieri & Matteo Dellepiane

Authors
  1. Ruggero Pintus
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Enrico Gobbetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Marco Callieri
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Matteo Dellepiane
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Enrico Gobbetti .

Editor information

Editors and Affiliations

  1. CNR-IBAM Institute of Archaeological and Monumental Heritage, Tito Scalo, Potenza, Italy

    Nicola Masini

  2. Institute for Electromagnetic Sensing of the Environment (CNR-IREA), Napoli, Italy

    Francesco Soldovieri

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Pintus, R., Gobbetti, E., Callieri, M., Dellepiane, M. (2017). Techniques for Seamless Color Registration and Mapping on Dense 3D Models. In: Masini, N., Soldovieri, F. (eds) Sensing the Past. Geotechnologies and the Environment, vol 16. Springer, Cham. https://doi.org/10.1007/978-3-319-50518-3_17

Download citation

Publish with us

Policies and ethics

Search

Navigation