Skip to main content
SpringerLink
Log in

Visualization/AR/VR/MR Systems

  • Chapter
  • First Online:
Active Lighting and Its Application for Computer Vision

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

  • 771 Accesses

Abstract

As is shown in previous chapters, active lighting is a powerful tool that satisfies various demands. Succeeding chapters provide several application examples of active lighting. Projecting active-lighting images onto real objects augments their appearance. Such projection mapping is now widely used in various fields, including the entertainment industry. Actively lighting multispectral lights onto oil paintings enables novel art modifications. Multispectral light can deceive human eyes because of our RGB limitations, because the light is represented using continuous wavelengths. Therefore, multispectral light can augment the visualization of real objects. Active lighting also enables us to capture the depth of the human body. Thus, a human pose can be estimated from the analysis of its depth. To represent a digital character, capturing actual human motion enables realistic duplication. Estimating human body positions is necessary for representing the motion of digital character and user positions in augmented-, virtual-, and mixed-reality systems.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Raskar R, Welch G, Low KL, Bandyopadhyay D (2001) Shader lamps: animating real objects with image-based illumination. In: Rendering techniques. Springer, pp 89–102

    Google Scholar 

  2. Mine MR, Van Baar J, Grundhofer A, Rose D, Yang B (2012) Projection-based augmented reality in disney theme parks. Computer 45(7):32–40

    Article  Google Scholar 

  3. Bimber O, Gatesy SM, Witmer LM, Raskar R, Encarnação LM (2002) Merging fossil specimens with computer-generated information. Computer 9:25–30

    Article  Google Scholar 

  4. Mukaigawa Y, Nishiyama M, Shakunaga T (2004) Virtual photometric environment using projector. In: Proceedings of the international conference on virtual systems and multimedia, pp 544–553

    Google Scholar 

  5. Hamasaki T, Itoh Y, Hiroi Y, Iwai D, Sugimoto M (2018) Hysar: hybrid material rendering by an optical see-through head-mounted display with spatial augmented reality projection. IEEE Trans Vis Comput Graph 24(4):1457–1466

    Article  Google Scholar 

  6. Raskar R, Beardsley P, van Baar J, Wang Y, Dietz P, Lee J, Leigh D, Willwacher T (2004) Rfig lamps: interacting with a self-describing world via photosensing wireless tags and projectors. In: ACM transactions on graphics (TOG), vol 23. ACM, pp 406–415

    Google Scholar 

  7. Hiura S, Tojo K, Inokuchi S (2003) 3-d tele-direction interface using video projector. In: ACM SIGGRAPH 2003 sketches & applications. ACM, pp 1–1

    Google Scholar 

  8. Kawabe T, Fukiage T, Sawayama M, Nishida S (2016) Deformation lamps: a projection technique to make static objects perceptually dynamic. ACM Trans Appl Percept (TAP) 13(2):10

    Google Scholar 

  9. Punpongsanon P, Iwai D, Sato K (2018) Flexeen: visually manipulating perceived fabric bending stiffness in spatial augmented reality. IEEE Trans Vis Comput Graph

    Google Scholar 

  10. Iwai D, Aoki M, Sato K (2018) Non-contact thermo-visual augmentation by IR-RGB projection. IEEE Trans Vis Comput Graph 25(4):1707–1716

    Article  Google Scholar 

  11. Punpongsanon P, Iwai D, Sato K (2015) Projection-based visualization of tangential deformation of nonrigid surface by deformation estimation using infrared texture. Virtual Real 19(1):45–56

    Article  Google Scholar 

  12. Punpongsanon P, Iwai D, Sato K (2015) Softar: visually manipulating haptic softness perception in spatial augmented reality. IEEE Trans Vis Comput Graph 21(11):1279–1288

    Article  Google Scholar 

  13. Kanamori T, Iwai D, Sato K (2018) Pseudo-shape sensation by stereoscopic projection mapping. IEEE Access 6:40649–40655

    Article  Google Scholar 

  14. Mine R, Iwai D, Hiura S, Sato K (2017) Shape optimization of fabricated transparent layer for pixel density uniformalization in non-planar rear projection. In: Proceedings of the 1st annual ACM symposium on computational fabrication. ACM, p 16

    Google Scholar 

  15. Asayama H, Iwai D, Sato K (2017) Fabricating diminishable visual markers for geometric registration in projection mapping. IEEE Trans Vis Comput Graph 24(2):1091–1102

    Article  Google Scholar 

  16. Amano T (2012) Shading illusion: a novel way for 3-d representation on paper media. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) workshops. IEEE, pp 1–6

    Google Scholar 

  17. Okutani N, Takezawa T, Iwai D, Sato K (2018) Stereoscopic capture in projection mapping. IEEE Access 6:65894–65900

    Article  Google Scholar 

  18. Park JI, Lee MH, Grossberg MD, Nayar SK (2007) Multispectral imaging using multiplexed illumination. In: Proceedings of the international conference on computer vision (ICCV). IEEE, pp 1–8

    Google Scholar 

  19. Bala R, Braun KM, Loce RP (2009) Watermark encoding and detection using narrowband illumination. In: Color and imaging conference, vol 2009. Society for Imaging Science and Technology, pp 139–142

    Google Scholar 

  20. Miyazaki D, Nakamura M, Baba M, Furukawa R, Hiura S (2016) Optimization of led illumination for generating metamerism. J Imaging Sci Technol 60(6):1–60502

    Article  Google Scholar 

  21. Jancosek M, Pajdla T (2011) Multi-view reconstruction preserving weakly-supported surfaces. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 3121–3128

    Google Scholar 

  22. Furukawa Y, Ponce J (2009) Accurate, dense, and robust multiview stereopsis. IEEE Trans Pattern Anal Mach Intell (PAMI) 32(8):1362–1376

    Article  Google Scholar 

  23. Besl PJ, McKay ND (1992) Method for registration of 3-d shapes. In: Sensor fusion IV: control paradigms and data structures, vol 1611. International Society for Optics and Photonics, pp 586–606

    Google Scholar 

  24. Izadi S, Kim D, Hilliges O, Molyneaux D, Newcombe R, Kohli P, Shotton J, Hodges S, Freeman D, Davison A, Fitzgibbon A (2011) Kinectfusion: real-time 3d reconstruction and interaction using a moving depth camera. In: Proceedings of the ACM symposium on user interface software and technology (UIST), pp 559–568

    Google Scholar 

  25. Newcombe RA, Izadi S, Hilliges O, Molyneaux D, Kim D, Davison AJ, Kohi P, Shotton J, Hodges S, Fitzgibbon A (2011) Kinectfusion: real-time dense surface mapping and tracking. In: 2011 10th IEEE international symposium on mixed and augmented reality. IEEE, pp 127–136

    Google Scholar 

  26. Whelan T, McDonald J, Kaess M, Fallon M, Johannsson H, Leonard J (2010) Kintinuous: Spatially extended KinectFusion. In: Proceedings of the RSS workshop on RGB-D: advanced reasoning with depth cameras

    Google Scholar 

  27. Amberg B, Romdhani S, Vetter T (2007) Optimal step nonrigid ICP algorithms for surface registration. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 1–8

    Google Scholar 

  28. Li H, Sumner RW, Pauly M (2008) Global correspondence optimization for non-rigid registration of depth scans, pp 1421–1430

    Google Scholar 

  29. Zollhöfer M, Nießner M, Izadi S, Rehmann C, Zach C, Fisher M, Wu C, Fitzgibbon A, Loop C, Theobalt C et al (2014) Real-time non-rigid reconstruction using an RGB-D camera. ACM Trans Graph (TOG) 33(4):1–12

    Article  Google Scholar 

  30. Newcombe RA, Fox D, Seitz SM (2015) Dynamicfusion: reconstruction and tracking of non-rigid scenes in real-time. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 343–352

    Google Scholar 

  31. Li H, Adams B, Guibas LJ, Pauly, M.: Robust single-view geometry and motion reconstruction. ACM Trans Graph (TOG) 28(5):175:1–175:10

    Google Scholar 

  32. Anguelov D, Srinivasan P, Koller D, Thrun S, Rodgers J, Davis J (2005) Scape: shape completion and animation of people. In: ACM transactions on graphics (TOG), vol 24. ACM, pp 408–416

    Google Scholar 

  33. Chen Y, Liu Z, Zhang Z (2013) Tensor-based human body modeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 105–112

    Google Scholar 

  34. Bogo F, Black MJ, Loper M, Romero J (2015) Detailed full-body reconstructions of moving people from monocular RGB-D sequences. In: Proceedings of the international conference on computer vision (ICCV), pp 2300–2308

    Google Scholar 

  35. Loper M, Mahmood N, Romero J, Pons-Moll G, Black MJ (2015) Smpl: a skinned multi-person linear model. ACM Trans Graph (TOG) 34(6):1–16

    Article  Google Scholar 

  36. Pishchulin L, Wuhrer S, Helten T, Theobalt C, Schiele B (2015) Building statistical shape spaces for 3d human modeling, pp 1–10. arXiv:1503.05860

  37. Kimura R, Sayo A, Dayrit FL, Nakashima Y, Kawasaki H, Blanco A, Ikeuchi K (2018) Representing a partially observed non-rigid 3d human using eigen-texture and eigen-deformation. In: Proceedings of the international conference on pattern recognition (ICPR). IEEE, pp 1043–1048

    Google Scholar 

  38. Sayo A, Onizuka H, Thomas D, Nakashima Y, Kawasaki H, Ikeuchi K (2019) Human shape reconstruction with loose clothes from partially observed data by pose specific deformation. In: Pacific-Rim symposium on image and video technology (PSVIT). Springer, pp 225–239

    Google Scholar 

  39. Nishino K, Sato Y, Ikeuchi K (2002) Eigen-texture method: appearance compression and synthesis based on a 3d model. IEEE Trans Pattern Anal Mach Intell (PAMI) 23(11):1257–1265

    Article  Google Scholar 

  40. Nakashima Y, Okura F, Kawai N, Kimura R, Kawasaki H, Ikeuchi K, Blanco A (2017) Realtime novel view synthesis with eigen-texture regression. In: Proceedings of the British machine vision conference (BMVC)

    Google Scholar 

  41. Bogo F, Kanazawa A, Lassner C, Gehler P, Romero J, Black MJ (2016) Keep it smpl: automatic estimation of 3d human pose and shape from a single image. In: Proceedings of the European conference on computer vision (ECCV). Springer, pp 561–578

    Google Scholar 

  42. Sorkine O, Cohen-Or D, Lipman Y, Alexa M, Rössl C, Seidel HP (2004) Laplacian surface editing. In: Proceedings of the 2004 Eurographics/ACM SIGGRAPH symposium on Geometry processing, pp 175–184

    Google Scholar 

  43. Nealen A, Igarashi T, Sorkine O, Alexa M (2006) Laplacian mesh optimization. In: Proceedings of the 4th international conference on computer graphics and interactive techniques in Australasia and Southeast Asia, pp 381–389

    Google Scholar 

  44. Yang J, Franco JS, Hétroy-Wheeler F, Wuhrer S (2018) Analyzing clothing layer deformation statistics of 3d human motions. In: Proceedings of the European conference on computer vision (ECCV), pp 237–253

    Google Scholar 

  45. Vlasic D, Baran I, Matusik W, Popović J (2008) Articulated mesh animation from multi-view silhouettes. ACM Trans Graph (TOG) 27(3):97:1–97:9. https://doi.org/10.1145/1360612.1360696

  46. Yu T, Zheng Z, Guo K, Zhao J, Dai Q, Li H, Pons-Moll G, Liu Y (2018) Doublefusion: real-time capture of human performances with inner body shapes from a single depth sensor. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 7287–7296

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Applied Robotics Research, Microsoft Corporation, Redmond, WA, USA

    Katsushi Ikeuchi

  2. Graduate School of Information Science and Technology, Osaka University, Osaka, Japan

    Yasuyuki Matsushita

  3. Artificial Intelligence Research Center, National Institute of Advanced Industrial Science and Technology, Tsukuba, Ibaraki, Japan

    Ryusuke Sagawa

  4. Department of Advanced Information Technology, Kyushu University, Fukuoka, Japan

    Hiroshi Kawasaki

  5. Division of Information Science, Nara Institute of Science and Technology, Nara, Japan

    Yasuhiro Mukaigawa

  6. Faculty of Information Sciences, Hiroshima City University, Hiroshima, Japan

    Ryo Furukawa

  7. Department of Intelligent Systems, Hiroshima City University, Hiroshima, Japan

    Daisuke Miyazaki

Authors
  1. Katsushi Ikeuchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yasuyuki Matsushita
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ryusuke Sagawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hiroshi Kawasaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yasuhiro Mukaigawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ryo Furukawa
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Daisuke Miyazaki
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Katsushi Ikeuchi .

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Ikeuchi, K. et al. (2020). Visualization/AR/VR/MR Systems. In: Active Lighting and Its Application for Computer Vision. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-030-56577-0_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-56577-0_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-56576-3

  • Online ISBN: 978-3-030-56577-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation