Skip to main content
SpringerLink
Log in

Augmented reality on cloth with realistic illumination

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Augmented reality (AR) is the concept of inserting virtual objects into real scenes. Often, augmentations are aligned with rigid planar objects in the scene. However, a more difficult task is to align non-rigid augmentations with flexible objects like cloth. To address this problem, we present a method to perform real-time flexible augmentations on cloth. Our method involves sparse cloth-tracking in video images using a new vision-based marker system with temporal coherence. We include an image-based method to automatically acquire real world illumination and shadows from the input frame. Non-rigid augmentations are achieved by rendering a textured 2D mesh aligned with the cloth surface, and combining the illumination result. The ability to perform realistic augmentations on cloth leads to applications in fashion, advertising, home decor and entertainment. We demonstrate our cloth augmentations with an application to interactively design T-shirts by demonstrating different virtual logos on a physical shirt in real-time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. ARToolKit, http://www.hitl.washington.edu/artoolkit

  2. Ballester C., Bertalmio M., Caselles V., Sapiro G. and Verdera J. (2001). Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. on Image Process. 10(8): 1200–1211

    Article  MATH  MathSciNet  Google Scholar 

  3. Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: K. Akeley (ed.) Siggraph 2000, Computer Graphics Proceedings, pp. 417–424. ACM Press/ACM SIGGRAPH/Addison Wesley Longman, New York (2000). http://citeseer.ist.psu.edu/bertalmio00image.html

  4. Billinghurst M., Kato H. and Poupyrev I. (2001). The magicbook—moving seamlessly between reality and virtuality. IEEE Comput. Graph. Appl. 21(3): 6–8

    Google Scholar 

  5. Bradley, D., Roth, G.: Augmenting non-rigid objects with realistic lighting. Technical Report NRC 47398, National Research Council of Canada (2004)

  6. Bradley D. and Roth G. (2007). Adaptive thresholding using the integral image. ACM J. Graph. Tools 12(2): 13–21

    Google Scholar 

  7. Bridson, R., Marino, S., Fedkiw, R.: Simulation of clothing with folds and wrinkles. In: SCA ’03: Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, pp. 28–36. Eurographics Association, Aire-la-Ville, Switzerland, Switzerland (2003)

  8. Comport, A.I., Marchand, E., Chaumette, F.: A real-time tracker for markerless augmented reality. In: IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2003), pp. 36–45. Tokyo, Japan (2003)

  9. Cormen T.H., Stein C., Rivest R.L. and Leiserson C.E. (2001). Introduction to Algorithms. McGraw-Hill Higher Education, New York

    MATH  Google Scholar 

  10. Debevec, P.: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. In: Proceedings of the 25th annual conference on Computer graphics and interactive techniques, pp. 189–198. ACM Press, New York (1998). http://doi.acm.org/10.1145/280814.280864

  11. Drettakis, G., Robert, L., Bougnoux, S.: Interactive common illumination for computer augmented reality. In: Proceedings of the Eurographics Workshop on Rendering Techniques ’97, pp. 45–56. Springer, Heidelberg (1997)

  12. Ehara, J., Saito, H.: Texture overlay for virtual clothing based on pca of silhouettes. In: IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2006) (2006)

  13. Fournier, A., Gunawan, A.S., Romanzin, C.: Common illumination between real and computer generated scenes. In: Proceedings of Graphics Interface ’93, pp. 254–262. Toronto, ON, Canada (1993)

  14. Genc, Y., Riedel, S., Souvannavong, F., Akinlar, C., Navab, N.: Marker-less tracking for ar: a learning-based approach. In: IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2002), pp. 295–304, Darmstadt, Germany (2002)

  15. Gibson, S., Cook, J., Howard, T., Hubbold, R.: Rapid shadow generation in real-world lighting environments. In: Proceedings of the 14th Eurographics workshop on Rendering, pp. 219–229. Eurographics Association (2003)

  16. Gibson, S., Murta, A.: Interactive rendering with real-world illumination. In: Proceedings of the Eurographics Workshop on Rendering Techniques 2000, pp. 365–376. Springer, Heidelberg (2000)

  17. Guskov, I.: Efficient tracking of regular patterns on non-rigid geometry. In: Proceedings of ICPR, pp. 1057–1060 (2002)

  18. Guskov, I., Klibanov, S., Bryant, B.: Trackable surfaces. In: ACM/EG Symposium on Computer Animation (2003)

  19. Haller, M., Drab, S., Hartmann, W.: A real-time shadow approach for an augmented reality application using shadow volumes. In: ACM Symposium on Virtual Reality Software and Technology (VRST 2003), pp. 56–65 (2003)

  20. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge ISBN: 0521623049 (2000)

  21. Madsen, C.B., Sørensen, M.K.D., Vittrup, M.: The importance of shadows in augmented reality.In: Proceedings: 6th Annual International Workshop on Presence, Aalborg, Denmark (2003)

  22. Okabe A., Boots B., Sugihara K. and Chiu S.N. (2000). Spatial Tessellations: Concepts and Applications of Voronoi Diagrams, 2nd Edn. Wiley, New York

    Google Scholar 

  23. Pilet, J., Lepetit, V., Fua, P.: Augmenting deformable objects in real-time. In: ISMAR ’05: Proceedings of the Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 134–137. IEEE Computer Society, Los Alamitos (2005)

  24. Pilet, J., Lepetit, V., Fua, P.: Real-time non-rigid surface detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2005), pp. 822–828 (2005)

  25. Pritchard D. and Heidrich W. (2003). Cloth motion capture. Computer Graphics Forum (Eurographics 2003) 22(3): 263–271

    Article  Google Scholar 

  26. Scholz, V., Magnor, M.: Texture replacement of garments in monocular video sequences. In: Heidrich, W., Akenine-Moeller, T. (eds.) Rendering Techniques 2006, Eurographics Symposium on Rendering, pp. 305–312. Eurographics (EG), ACM SIGGRAPH, Eurographics, Nicosia, Cyprus (2006)

  27. Scholz, V., Magnor, M.A.: Cloth motion from optical flow. In: Girod, B., Magnor, M., Seidel H.P. (eds.) Proceedings of Vision, Modeling and Visualization 2004, pp. 117–124 (2004)

  28. Scholz V., Stich T., Keckeisen M., Wacker M. and Magnor M. (2005). Garment motion capture using color-coded patterns. Computer Graphics Forum (Proc. Eurographics EG’05) 24(3): 439–448

    Article  Google Scholar 

  29. Simon, G., Fitzgibbon, A.W., Zisserman, A.: Markerless tracking using planar structures in the scene. In: Proceedings of International Symposium on Augmented Reality, pp. 120–128 (2000). http://www.robots.ox.ac.uk/vgg

  30. Stauder, J.: Augmented reality with automatic illumination control incorporating ellipsoidal models. IEEE Trans. Mult. 1(2), 136–143 (1999) http://citeseer.ist.psu.edu/stauder99augmented.html

    Google Scholar 

  31. White, R., Forsyth, D.: Retexturing single views using texture and shading. In: European Conference on Computer Vision, vol. LNCS 3954, pp. 70–81. Springer, Heidelberg (2006)

Download references

Author information

Author notes

  1. Derek Bradley

    Present address: University of British Columbia, Vancouver, BC, Canada

Authors and Affiliations

  1. Department of Computer Science, Carleton University, Ottawa, ON, Canada

    Derek Bradley & Prosenjit Bose

  2. Institute for Information Technology, National Research Council of Canada, Ottawa, ON, Canada

    Gerhard Roth

Authors
  1. Derek Bradley
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gerhard Roth
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Prosenjit Bose
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Derek Bradley.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bradley, D., Roth, G. & Bose, P. Augmented reality on cloth with realistic illumination. Machine Vision and Applications 20, 85–92 (2009). https://doi.org/10.1007/s00138-007-0108-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-007-0108-9

Keywords

Search

Navigation