The Visual Computer

, Volume 30, Issue 4, pp 417–429 | Cite as

Vision-based all-in-one solution for augmented reality and its storytelling applications

Original Article

Abstract

In this paper, we propose a vision-based all-in-one solution for augmented reality where users want to exploit unknown 3D objects in their systems. In our solution, we facilitate two time-consuming off-line processes: obtaining information, such as keyframes and keypoints, for real-time tracking of unknown 3D targets, and estimating local coordinates with a scale for accurate registration of virtual content. The proposed solution only requires images with minimal interactions. The users do not need to know about 3D markerless tracking in depth. At the end, we propose a framework for AR miniatures systems to verify the effectiveness of our solution. As a result, all developed systems worked in real-time, more than 25 fps, and showed reliable registration even in severe viewpoint changes. Our demonstration videos are available in the supplemental materials.

Keywords

Augmented reality Markerless tracking Miniature Framework Storytelling 

Supplementary material

(M4V 7.6 MB)

(M4V 9.5 MB)

(M4V 6.2 MB)

References

  1. 1.
    Aittala, M.: Inverse lighting and photorealistic rendering for augmented reality. Vis. Comput. 26, 669–678 (2010) CrossRefGoogle Scholar
  2. 2.
    Bastian, J., Ward, B., Hill, R., Van Den Hengel, A., Dick, A.: Interactive modelling for AR applications. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 199–205 (2010) Google Scholar
  3. 3.
    Castle, R., Murray, D.: Keyframe-based recognition and localization during video-rate parallel tracking and mapping. Image Vis. Comput. 29(8), 524–532 (2011) CrossRefGoogle Scholar
  4. 4.
    Castle, R.O., Murray, D.W.: Object recognition and localization while tracking and mapping. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 179–180 (2009) Google Scholar
  5. 5.
    Correia, N., Romero, L.: Storing user experiences in mixed reality using hypermedia. Vis. Comput. 22(12), 991–1001 (2006) CrossRefGoogle Scholar
  6. 6.
    Dong, Z., Zhang, G., Jia, J., Bao, H.: Keyframe-based real-time camera tracking. In: IEEE International Conference on Computer Vision, Miami, USA, pp. 1538–1545 (2009) Google Scholar
  7. 7.
    Egges, A., Papagiannakis, G., Magnenat-Thalmann, N.: Presence and interaction in mixed reality environments. Vis. Comput. 23(5), 317–333 (2007) CrossRefGoogle Scholar
  8. 8.
    Fiala, M.: ARTag, a fiducial marker system using digital techniques. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 590–596 (2005) Google Scholar
  9. 9.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981) CrossRefMathSciNetGoogle Scholar
  10. 10.
    Moreno-Noguer, F., Lepetit, V., Fua, P.: Accurate non-iterative o(n) solution to the pnp problem. In: IEEE International Conference on Computer Vision, Rio de Janeiro, Brazil, pp. 1–8 (2007) Google Scholar
  11. 11.
    Gallup, D., Frahm, J.M., Pollefeys, M.: Piecewise planar and non-planar stereo for urban scene reconstruction. In: IEEE International Conference on Computer Vision, pp. 1418–1425 (2010) Google Scholar
  12. 12.
    Van den Hengel, A., Hill, R., Ward, B., Dick, A.: In situ image-based modeling. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 107–110 (2009) Google Scholar
  13. 13.
    Irschara, A., Zach, C., Frahm, J.M., Bischof, H.: From structure-from-motion point clouds to fast location recognition. In: IEEE International Conference on Computer Vision, Miami, USA, pp. 2599–2606 (2009) Google Scholar
  14. 14.
    Kato, H., Billinghurst, M.: Marker tracking and HMD calibration for a video-based augmented reality conferencing system. In: IEEE/ACM International Workshop on Augmented Reality, pp. 85–94 (1999) Google Scholar
  15. 15.
    Kim, K., Lepetit, V., Woo, W.: Scalable real-time planar targets tracking for digilog books. Vis. Comput. 26, 1145–1154 (2010) CrossRefGoogle Scholar
  16. 16.
    Kim, K., Park, Y., Woo, W.: Digilog miniature: real-time immersive, and interactive ar on miniatures. In: Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and Its Applications in Industry, VRCAI ’10, pp. 161–168. ACM, New York (2010) CrossRefGoogle Scholar
  17. 17.
    Klein, G., Murray, D.: Parallel tracking and mapping for small AR workspaces. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 83–86 (2007) Google Scholar
  18. 18.
    Lepetit, V., Lagger, P., Fua, P.: Randomized trees for real-time keypoint recognition. In: Proceedings of the Conference on Computer Vision and Pattern Recognition, pp. 775–781 (2005) Google Scholar
  19. 19.
    Lourakis, M.A., Argyros, A.: SBA: a software package for generic sparse bundle adjustment. ACM Trans. Math. Softw. 36(1), 1–30 (2009) CrossRefMathSciNetGoogle Scholar
  20. 20.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004) CrossRefGoogle Scholar
  21. 21.
    Newcombe, R., Davison, A.: Live dense reconstruction with a single moving camera. In: IEEE International Conference on Computer Vision, pp. 1498–1505 (2010) Google Scholar
  22. 22.
    Nister, D., Stewenius, H.: Scalable recognition with a vocabulary tree. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2161–2168 (2006) Google Scholar
  23. 23.
    Pilet, J., Geiger, A., Lagger, P., Lepetit, V., Fua, P.: An all-in-one solution to geometric and photometric calibration. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 69–78 (2006) CrossRefGoogle Scholar
  24. 24.
    Scherrer, C., Pilet, J., Fua, P., Lepetit, V.: The haunted book. In: International Symposium on Mixed and Augmented Reality, pp. 163–164 (2008) Google Scholar
  25. 25.
    Skrypnyk, I., Lowe, D.: Scene modelling, recognition and tracking with invariant image features. In: IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 110–119 (2004) CrossRefGoogle Scholar
  26. 26.
    Taketa, N., Hayashi, K., Kato, H., Noshida, S.: Virtual pop-up book based on augmented reality. In: Symposium on Human Interface 2007, Held as Part of HCI International 2007. Lecture Notes in Computer Science, pp. 475–484. Springer, Berlin (2007) Google Scholar
  27. 27.
    Wagner, D., Schmalstieg, D.: Artoolkitplus for pose tracking on mobile devices. In: Proceedings of 12th Computer Vision Winter Workshop (CVWW’07), pp. 139–146 (2007) Google Scholar
  28. 28.
    Wu, C.: SiftGPU: A GPU: a GPU implementation of scale invariant feature transform (SIFT) (2007). http://cs.unc.edu/~ccwu/siftgpu

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  1. 1.Qualcomm Austria Research Center GmbHViennaAustria
  2. 2.KAIST UVR Lab.DaejeonSouth Korea

Personalised recommendations