Skip to main content
SpringerLink
Log in

Vision-based all-in-one solution for augmented reality and its storytelling applications

  • Original Article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In this paper, we propose a vision-based all-in-one solution for augmented reality where users want to exploit unknown 3D objects in their systems. In our solution, we facilitate two time-consuming off-line processes: obtaining information, such as keyframes and keypoints, for real-time tracking of unknown 3D targets, and estimating local coordinates with a scale for accurate registration of virtual content. The proposed solution only requires images with minimal interactions. The users do not need to know about 3D markerless tracking in depth. At the end, we propose a framework for AR miniatures systems to verify the effectiveness of our solution. As a result, all developed systems worked in real-time, more than 25 fps, and showed reliable registration even in severe viewpoint changes. Our demonstration videos are available in the supplemental materials.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. 3DVIA Virtools, http://www.3ds.com/products/3dvia/3dvia-virtools/.

  2. Point Grey Research, http://www.ptgrey.com.

  3. Open Computer Vision Library, OpenCV, http://opencv.willowgarage.com.

References

  1. Aittala, M.: Inverse lighting and photorealistic rendering for augmented reality. Vis. Comput. 26, 669–678 (2010)

    Article  Google Scholar 

  2. Bastian, J., Ward, B., Hill, R., Van Den Hengel, A., Dick, A.: Interactive modelling for AR applications. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 199–205 (2010)

    Google Scholar 

  3. Castle, R., Murray, D.: Keyframe-based recognition and localization during video-rate parallel tracking and mapping. Image Vis. Comput. 29(8), 524–532 (2011)

    Article  Google Scholar 

  4. Castle, R.O., Murray, D.W.: Object recognition and localization while tracking and mapping. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 179–180 (2009)

    Google Scholar 

  5. Correia, N., Romero, L.: Storing user experiences in mixed reality using hypermedia. Vis. Comput. 22(12), 991–1001 (2006)

    Article  Google Scholar 

  6. Dong, Z., Zhang, G., Jia, J., Bao, H.: Keyframe-based real-time camera tracking. In: IEEE International Conference on Computer Vision, Miami, USA, pp. 1538–1545 (2009)

    Google Scholar 

  7. Egges, A., Papagiannakis, G., Magnenat-Thalmann, N.: Presence and interaction in mixed reality environments. Vis. Comput. 23(5), 317–333 (2007)

    Article  Google Scholar 

  8. Fiala, M.: ARTag, a fiducial marker system using digital techniques. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 590–596 (2005)

    Google Scholar 

  9. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  10. Moreno-Noguer, F., Lepetit, V., Fua, P.: Accurate non-iterative o(n) solution to the pnp problem. In: IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil, pp. 1–8 (2007)

    Google Scholar 

  11. Gallup, D., Frahm, J.M., Pollefeys, M.: Piecewise planar and non-planar stereo for urban scene reconstruction. In: IEEE International Conference on Computer Vision, pp. 1418–1425 (2010)

    Google Scholar 

  12. Van den Hengel, A., Hill, R., Ward, B., Dick, A.: In situ image-based modeling. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 107–110 (2009)

    Google Scholar 

  13. Irschara, A., Zach, C., Frahm, J.M., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: IEEE International Conference on Computer Vision, Miami, USA, pp. 2599–2606 (2009)

    Google Scholar 

  14. Kato, H., Billinghurst, M.: Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: IEEE/ACM International Workshop on Augmented Reality, pp. 85–94 (1999)

    Google Scholar 

  15. Kim, K., Lepetit, V., Woo, W.: Scalable real-time planar targets tracking for digilog books. Vis. Comput. 26, 1145–1154 (2010)

    Article  Google Scholar 

  16. Kim, K., Park, Y., Woo, W.: Digilog miniature: real-time immersive, and interactive ar on miniatures. In: Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry, VRCAI ’10, pp. 161–168. ACM, New York (2010)

    Chapter  Google Scholar 

  17. Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 83–86 (2007)

    Google Scholar 

  18. Lepetit, V., Lagger, P., Fua, P.: Randomized trees for real-time keypoint recognition. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 775–781 (2005)

    Google Scholar 

  19. Lourakis, M.A., Argyros, A.: SBA: a software package for generic sparse bundle adjustment. ACM Trans. Math. Softw. 36(1), 1–30 (2009)

    Article  MathSciNet  Google Scholar 

  20. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  21. Newcombe, R., Davison, A.: Live dense reconstruction with a single moving camera. In: IEEE International Conference on Computer Vision, pp. 1498–1505 (2010)

    Google Scholar 

  22. Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2161–2168 (2006)

    Google Scholar 

  23. Pilet, J., Geiger, A., Lagger, P., Lepetit, V., Fua, P.: An all-in-one solution to geometric and photometric calibration. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 69–78 (2006)

    Chapter  Google Scholar 

  24. Scherrer, C., Pilet, J., Fua, P., Lepetit, V.: The haunted book. In: International Symposium on Mixed and Augmented Reality, pp. 163–164 (2008)

    Google Scholar 

  25. Skrypnyk, I., Lowe, D.: Scene modelling, recognition and tracking with invariant image features. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 110–119 (2004)

    Chapter  Google Scholar 

  26. Taketa, N., Hayashi, K., Kato, H., Noshida, S.: Virtual pop-up book based on augmented reality. In: Symposium on Human Interface 2007, Held as Part of HCI International 2007. Lecture Notes in Computer Science, pp. 475–484. Springer, Berlin (2007)

    Google Scholar 

  27. Wagner, D., Schmalstieg, D.: Artoolkitplus for pose tracking on mobile devices. In: Proceedings of 12th Computer Vision Winter Workshop (CVWW’07), pp. 139–146 (2007)

    Google Scholar 

  28. Wu, C.: SiftGPU: A GPU: a GPU implementation of scale invariant feature transform (SIFT) (2007). http://cs.unc.edu/~ccwu/siftgpu

Download references

Acknowledgements

This work was supported by the Global Frontier R&D Program on 〈Humancentered Interaction for Coexistence〉 funded by the National Research Foundation of Korea grant funded by the Korean Government (MSIP) (NRF-2010-0029751). And also this work was supported by the DigiLog Miniature Augmented Reality Research Program funded by KAIST Research Foundation.

Author information

Authors and Affiliations

  1. Qualcomm Austria Research Center GmbH, Vienna, Austria

    Kiyoung Kim

  2. KAIST UVR Lab., Daejeon, South Korea

    Noh-young Park & Woontack Woo

Authors
  1. Kiyoung Kim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Noh-young Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Woontack Woo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Woontack Woo.

Electronic Supplementary Material

Below are the links to the electronic supplementary material.

(M4V 7.6 MB)

(M4V 9.5 MB)

(M4V 6.2 MB)

Rights and permissions

Reprints and permissions

About this article

Cite this article

Kim, K., Park, Ny. & Woo, W. Vision-based all-in-one solution for augmented reality and its storytelling applications. Vis Comput 30, 417–429 (2014). https://doi.org/10.1007/s00371-013-0865-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-013-0865-6

Keywords

Search

Navigation