Advertisement

Color Correction for Image-Based Modeling in the Large

  • Tianwei Shen
  • Jinglu Wang
  • Tian FangEmail author
  • Siyu Zhu
  • Long Quan
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 10114)

Abstract

Current texture creation methods for image-based modeling suffer from color discontinuity issues due to drastically varying conditions of illumination, exposure and time during the image capturing process. This paper proposes a novel system that generates consistent textures for triangular meshes. The key to our system is a color correction framework for large-scale unordered image collections. We model the problem as a graph-structured optimization over the overlapping regions of image pairs. After reconstructing the mesh of the scene, we accurately calculate matched image regions by re-projecting images onto the mesh. Then the image collection is robustly adjusted using a non-linear least square solver over color histograms in an unsupervised fashion. Finally, a connectivity-preserving edge pruning method is introduced to accelerate the color correction process. This system is evaluated with crowdsourcing image collections containing medium-sized scenes and city-scale urban datasets. To the best of our knowledge, this system is the first consistent texturing system for image-based modeling that is capable of handling thousands of input images.

Keywords

Color Histogram Image Collection Color Correction Scene Graph Color Tone 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Notes

Acknowledgement

The authors would like to thank all the anonymous reviewers for their constructive feedbacks. This work is supported by Hong Kong RGC 16208614, T22-603/15N, Hong Kong ITC PSKL12EG02, and China 973 program, 2012CB316300.

Supplementary material

416263_1_En_24_MOESM1_ESM.pdf (851 kb)
Supplementary material 1 (pdf 851 KB)

References

  1. 1.
    Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74, 59–73 (2007)CrossRefGoogle Scholar
  2. 2.
    Eden, A., Uyttendaele, M., Szeliski, R.: Seamless image stitching of scenes with large motions and exposure differences. In: Computer Vision and Pattern Recognition (CVPR), pp. 2498–2505 (2006)Google Scholar
  3. 3.
    Xiong, Y., Pulli, K.: Color matching of image sequences with combined gamma and linear corrections. In: International Conference on ACM Multimedia, pp. 261–270 (2010)Google Scholar
  4. 4.
    Moulon, P., Duisit, B., Monasse, P.: Global multiple-view color consistency. In: Conference on Visual Media Production (CVMP) (2013)Google Scholar
  5. 5.
    Reinhard, E., Ashikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21, 34–41 (2001)CrossRefGoogle Scholar
  6. 6.
    Ruderman, D.L., Cronin, T.W., Chiao, C.C.: Statistics of cone responses to natural images: implications for visual coding. JOSA A 15, 2036–2045 (1998)CrossRefGoogle Scholar
  7. 7.
    Tian, G.Y., Gledhill, D., Taylor, D., Clarke, D.: Colour correction for panoramic imaging. In: International Conference on Information Visualisation, pp. 483–488 (2002)Google Scholar
  8. 8.
    Hwang, Y., Lee, J.Y., Kweon, I.S., Kim, S.J.: Color transfer using probabilistic moving least squares. In: Computer Vision and Pattern Recognition (CVPR), pp. 3342–3349 (2014)Google Scholar
  9. 9.
    Nguyen, R., Kim, S., Brown, M.: Illuminant aware gamut-based color transfer. Comput. Graph. Forum 7, 319–328 (2014)CrossRefGoogle Scholar
  10. 10.
    Xiao, X., Ma, L.: Gradient-preserving color transfer. Comput. Graph. Forum 7, 1879–1886 (2009)CrossRefGoogle Scholar
  11. 11.
    Xu, W., Mulligan, J.: Performance evaluation of color correction approaches for automatic multi-view image and video stitching. In: Computer Vision and Pattern Recognition (CVPR), pp. 263–270 (2010)Google Scholar
  12. 12.
    Nanda, H., Cutler, R.: Practical calibrations for a real-time digital omnidirectional camera. CVPR Technical Sketch (2001)Google Scholar
  13. 13.
    Burt, P.J., Adelson, E.H.: A multiresolution spline with application to image mosaics. ACM Trans. Graph. (TOG) 2, 217–236 (1983)CrossRefGoogle Scholar
  14. 14.
    Yamamoto, K., Oi, R.: Color correction for multi-view video using energy minimization of view networks. Int. J. Autom. Comput. 5, 234–245 (2008)CrossRefGoogle Scholar
  15. 15.
    Liu, Z., Marlet, R.: Virtual line descriptor and semi-local matching method for reliable feature correspondence. In: British Machine Vision Conference (BMVC) (2012)Google Scholar
  16. 16.
    Waechter, M., Moehrle, N., Goesele, M.: Let there be color! large-scale texturing of 3D reconstructions. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 836–850. Springer, Heidelberg (2014)Google Scholar
  17. 17.
    Allène, C., Pons, J.P., Keriven, R.: Seamless image-based texture atlases using multi-band blending. In: International Conference on Pattern Recognition (ICPR), pp. 1–4 (2008)Google Scholar
  18. 18.
    HaCohen, Y., Shechtman, E., Goldman, D.B., Lischinski, D.: Optimizing color consistency in photo collections. ACM Trans. Graph. (TOG) 32, 38 (2013)CrossRefGoogle Scholar
  19. 19.
    Agarwal, S., Snavely, N., Simon, I., Seitz, S.M., Szeliski, R.: Building Rome in a day. In: International Conference on Computer Vision (ICCV), pp. 72–79 (2009)Google Scholar
  20. 20.
    Shen, T., Zhu, S., Fang, T., Zhang, R., Quan, L.: Graph-based consistent matching for structure-from-motion. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 139–155. Springer, Heidelberg (2016)CrossRefGoogle Scholar
  21. 21.
    Goesele, M., Snavely, N., Curless, B., Hoppe, H., Seitz, S.M.: Multi-view stereo for community photo collections. In: International Conference on Computer Vision (ICCV), pp. 1–8 (2007)Google Scholar
  22. 22.
    Furukawa, Y., Curless, B., Seitz, S.M., Szeliski, R.: Towards internet-scale multi-view stereo. In: Computer Vision and Pattern Recognition (CVPR), pp. 1434–1441 (2010)Google Scholar
  23. 23.
    Kazhdan, M., Bolitho, M., Hoppe, H.: Poisson surface reconstruction. In: Eurographics Symposium on Geometry Processing (2006)Google Scholar
  24. 24.
    Levenberg, K.: A method for the solution of certain non-linear problems in least squares (1944)Google Scholar
  25. 25.
    Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11, 431–441 (1963)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Zhang, M., Georganas, N.D.: Fast color correction using principal regions mapping in different color spaces. Real-Time Imaging 10, 23–30 (2004)CrossRefGoogle Scholar
  27. 27.
    Zhou, F., Mahler, S., Toivonen, H.: Simplification of networks by edge pruning. In: Berthold, M.R. (ed.) Bisociative Knowledge Discovery. LNCS (LNAI), vol. 7250, pp. 179–198. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-31830-6_13 CrossRefGoogle Scholar
  28. 28.
    Wright, S., Holt, J.N.: An inexact Levenberg-Marquardt method for large sparse nonlinear least squares. J. Aust. Math. Soc. Ser. B. Appl. Math. 26, 387–403 (1985)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from internet photo collections. Int. J. Comput. Vis. 80, 189–210 (2008)CrossRefGoogle Scholar
  30. 30.
    Moulon, P., Monasse, P., Marlet, R.: Global fusion of relative motions for robust, accurate and scalable structure from motion. In: International Conference on Computer Vision (ICCV), pp. 3248–3255 (2013)Google Scholar
  31. 31.
    Furukawa, Y., Ponce, J.: Accurate, dense, and robust multiview stereopsis. Pattern Anal. Mach. Intell. (PAMI) 32, 1362–1376 (2010)CrossRefGoogle Scholar
  32. 32.
    Lhuillier, M., Quan, L.: A quasi-dense approach to surface reconstruction from uncalibrated images. Pattern Anal. Mach. Intell. (PAMI) 27, 418–433 (2005)CrossRefGoogle Scholar
  33. 33.
    Agarwal, S., Mierle, K., Others: Ceres solver. (http://ceres-solver.org)
  34. 34.
    Strecha, C., von Hansen, W., Gool, L.V., Fua, P., Thoennessen, U.: On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: Computer Vision and Pattern Recognition (CVPR), pp. 1–8 (2008)Google Scholar
  35. 35.
    Lempitsky, V., Ivanov, D.: Seamless mosaicing of image-based texture maps. In: Computer Vision and Pattern Recognition (CVPR), pp. 1–6 (2007)Google Scholar
  36. 36.
    Tan, P., Lin, S., Quan, L., Shum, H.Y.: Highlight removal by illumination-constrained inpainting. In: International Conference on Computer Vision (ICCV), pp. 164–169 (2003)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Tianwei Shen
    • 1
  • Jinglu Wang
    • 1
  • Tian Fang
    • 1
    Email author
  • Siyu Zhu
    • 1
  • Long Quan
    • 1
  1. 1.Department of Computer Science and EngineeringHong Kong University of Science and TechnologyKowloonHong Kong

Personalised recommendations