Radiometric compensation for non-rigid surfaces by continuously estimating inter-pixel correspondence

  • 55 Accesses


In recent years, radiometric compensation techniques for realizing ideal image projection, even for a patterned surface, have attracted attention. In radiometric compensation, the influence of the pattern can be canceled based on the input and output relation, referred to as the response function, which is determined by a projector–camera system. However, since the response function strongly depends on the inter-pixel correspondence between the projector and the camera, the projection surface is restricted to being a rigid body. In the present study, we achieve radiometric compensation for a non-rigid surface, such as a swaying curtain in a normal room, by estimating the inter-pixel correspondence in real time. The reflectance of the projection surface is estimated based on observation of the projected image by the camera without using special equipment, and the offset of the correspondence is estimated based on its validity. We evaluate the effectiveness of the proposed method for various combinations of curtain patterns and projected images.

This is a preview of subscription content, log in to check access.

Access options

Buy single article

Instant unlimited access to the full article PDF.

US$ 39.95

Price includes VAT for USA

Subscribe to journal

Immediate online access to all issues from 2019. Subscription will auto renew annually.

US$ 199

This is the net price. Taxes to be calculated in checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11


  1. 1.

    Amano, T.: Projection center calibration for a co-located projector camera system. Comput. Vis. Pattern Recognit. 2014, 443–448 (2014)

  2. 2.

    Ashdown, M., Okabe, T., Sato, I., Sato, Y.: Robust content-dependent photometric projector compensation. In: 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW’06), p. 6 (2006)

  3. 3.

    Bimber, O., Emmerling, A., Klemmer, T.: Embedded entertainment with smart projectors. IEEE Comput. 38(1), 48–55 (2005)

  4. 4.

    Bimber, O., Iwai, D., Wetzstein, G., Grundhöfer, A.: The visual computing of projector-camera systems. In: ACM SIGGRAPH 2008 Classes, SIGGRAPH’08, pp. 84:1–84:25. ACM (2008)

  5. 5.

    Bimber, O., Rasker, R.: Spatial Augmented Reality: Merging Real and Virtual Worlds. A.K. Peters Ltd., Natick (2005)

  6. 6.

    Cotting, D., Naef, M., Gross, M., Fuchs, H.: Embedding imperceptible patterns into projected images for simultaneous acquisition and display. In: Third IEEE and ACM International Symposium on Mixed and Augmented Reality (2004)

  7. 7.

    Fujii, K., Grossberg, M.D., Nayar, S.K.: A projector–camera system with real-time photometric adaptation for dynamic environments. Comput. Vis. Pattern Recognit. 2005, 814–821 (2005)

  8. 8.

    Grundhöfer, A., Bimber, O.: Real-time adaptive radiometric compensation. IEEE Trans. Vis. Comput. Graph. 14(1), 97–108 (2008)

  9. 9.

    Grundhöfer, A., Iwai, D.: Robust, error-tolerant photometric projector compensation. IEEE Trans. Image Process. 24(12), 5086–5099 (2015)

  10. 10.

    Hashimoto, N., Kosaka, K.: Photometric compensation for practical and complex textures. In: ACM SIGGRAPH2015 Posters (2015)

  11. 11.

    Hashimoto, N., Kosaka, K.: Continuous photometric compensation for deformable objects. In: ACM SIGGRAPH 2017 Posters, p. 72 (2017)

  12. 12.

    Hastings-James, R., Sage, M.W.: Recursive generalised-least-squares procedure for online identification of process parameters. Proc. Inst. Electr. Eng. 116(12), 2057–2062 (1969)

  13. 13.

    Lee, J.C., Dietz, P.H., Maynes-Aminzade, D., Raskar, R., Hudson, S.E.: Automatic projector calibration with embedded light sensors. In: The 17th Annual ACM Symposium on User Interface Software and Technology (2004)

  14. 14.

    Li, Y., Bao, N., Yuan, Q., Lu, D.: Real-time continuous geometric calibration for projector–camera system under ambient illumination. In: 2012 International Conference on Virtual Reality and Visualization, pp. 7–12 (2012)

  15. 15.

    Li, Y., Majumder, A., Gopi, M., Wang, C., Zhao, J.: Practical radiometric compensation for projection display on textured surfaces using a multidimensional model. Comput. Graph. Forum 37(2), 365–375 (2018)

  16. 16.

    Majumder, A., He, Z., Towles, H., Welch, G.: Achieving color uniformity across multi-projector displays. In: Proceedings Visualization 2000. VIS 2000 (Cat. No. 00CH37145), pp. 117–124 (2000)

  17. 17.

    Majumder, A., Stevens, R.: Color nonuniformity in projection-based displays: analysis and solutions. IEEE Trans. Vis. Comput. Graph. 10(2), 177–188 (2004)

  18. 18.

    Majumder, A., Stevens, R.: Perceptual photometric seamlessness in projection-based tiled displays. ACM Trans. Graph. 24(1), 118–139 (2005)

  19. 19.

    Mihara, S., Iwai, D., Sato, K.: Artifact reduction in radiometric compensation of projector–camera system for steep reflectance variations. IEEE Trans. Circuits Syst. Video Technol. 24(9), 1631–1638 (2014)

  20. 20.

    Nakamura, T., Watanabe, A., Hashimoto, N.: Dynamic projection mapping. In: ACM SIGGRAPH2012, Posters (2012)

  21. 21.

    Nayar, S.K., Peri, J., Grossberg, M., Belhumeur, P.: A projection system with radiometric compensation for screen imperfections. In: IEEE International Workshop on Projector–Camera Systems (2003)

  22. 22.

    Sajadi, B., Lazarov, M., Majumder, A.: Adict: accurate direct and inverse color transformation. In: Computer Vision—ECCV 2010, pp. 72–86. Springer, Berlin (2010)

  23. 23.

    Wang, D., Sato, I., Okabe, T., Sato, Y.: Radiometric compensation in a projector-camera system based properties of human vision system. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)—Workshops, pp. 100–100 (2005)

  24. 24.

    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

  25. 25.

    Watanabe, Y., Narita, G., Tatsuno, S., Yuasa, T., Sumino, K., Ishikawa, M.: High-speed 8-bit image projector at 1000 fps with 3 ms delay. In: The International Display Workshops (IDW2015), pp. 1064–1065 (2015)

  26. 26.

    Wetzstein, G., Bimber, O.: Radiometric compensation through inverse light transport. In: 15th Pacific Conference on Computer Graphics and Applications (PG’07), pp. 391–399 (2007)

  27. 27.

    Yoshida, T., Horii, C., Sato, K.: A virtual color reconstruction system for real heritage with light projection. In: VSMM: Proceedings of International Conference on Virtual Systems and Multimedia, pp. 1–7 (2003)

  28. 28.

    Zhang, S., Huang, P.: High-resolution, real-time 3D shape acquisition. In: 2004 Conference on Computer Vision and Pattern Recognition Workshop, pp. 28–28 (2004).

  29. 29.

    Zollmann, S., Bimber, O.: Imperceptible calibration for radiometric compensation. In: Cignoni, P., Sochor, J. (eds.) EG Short Papers. The Eurographics Association, Geneva (2007).

  30. 30.

    Zollmann, S., Langlotz, T., Bimber, O.: Passive-active geometric calibration for view-dependent projections onto arbitrary surfaces. JVRB J. Virtual Real. Broadcast. 4(6) (2007)

Download references


This study was funded by the Japan Society for the Promotion of Science, Kakenhi Grant Numbers JP16K00267 and JP19H04152.

Author information

Correspondence to Naoki Hashimoto.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Hashimoto, N., Yoshimura, K. Radiometric compensation for non-rigid surfaces by continuously estimating inter-pixel correspondence. Vis Comput (2020) doi:10.1007/s00371-019-01790-8

Download citation


  • Radiometric compensation
  • Inter-pixel correspondence
  • Non-rigid surface
  • Response function
  • ProCam system