Advertisement

A Method of Registering Virtual Objects in Monocular Augmented Reality System

  • Zeye Wu
  • Pengrui Wang
  • Wujun Che
Conference paper
Part of the Communications in Computer and Information Science book series (CCIS, volume 875)

Abstract

A flexible novel method of registering virtual objects in monocular AR system is presented in this paper. Monocular AR systems use SLAM-related techniques to obtain the camera pose, of which the translation component is on a random scale. We add a scale calibration process to acquire the scale factor from the SLAM map to the real world and provide a closed-form solution of the transformation between two coordinate systems with different scales. We also describe the framework of an AR system based on our method with implementation. The proposed system can easily initialize virtual objects’ position, orientation and size by using a known reference in the real scene and the reference is no longer needed in the later process. Our method is flexible, simple to set up and easy to control. The results show the proposed method can apply to real-time interactive AR applications.

Keywords

Augmented reality Monocular SLAM Virtual objects registration Scale calibration 

Notes

Acknowledgements

This work is supported by National Natural Science Foundation of China (No. 61471359) and National Key R&D Plan of China (No. 2016YFB1001404).

References

  1. 1.
    Kaufmann, H.: Construct3D: an augmented reality application for mathematics and geometry education. In: ACM Multimedia, pp. 656–657. (2002)Google Scholar
  2. 2.
    Bichlmeier, C., Sielhorst, T., Heining, S.M., Navab, N.: Improving depth perception in medical AR. In: Horsch, A., Deserno, T.M., Handels, H., Meinzer, H.P., Tolxdorff, T. (eds.) Bildverarbeitung für die Medizin 2007. Informatik aktuell, pp. 217–221. Springer, Heidelberg (2007).  https://doi.org/10.1007/978-3-540-71091-2_44CrossRefGoogle Scholar
  3. 3.
    Livingston, M.A., et al.: Military applications of augmented reality. In: Furht, B. (ed.) Handbook of Augmented Reality, pp. 671–706. Springer, New York (2011).  https://doi.org/10.1007/978-1-4614-0064-6_31CrossRefGoogle Scholar
  4. 4.
    Pucihar, K.C., Coulton, P.: Exploring the evolution of mobile augmented reality for future entertainment systems. Conf. Comput. Eur. 11, 1–16 (2013)CrossRefGoogle Scholar
  5. 5.
    Carvalho, C.V.D.A., Lemos, B.M.: Possibilities of augmented reality use in mathematics aiming at a meaningful learning. Creat. Educ. 05, 690–700 (2014)CrossRefGoogle Scholar
  6. 6.
    Feng, Z., Duh, H.B.L., Billinghurst, M.: Trends in augmented reality tracking, interaction and display: a review of ten years of ISMAR. In: 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality, pp. 193–202 (2008)Google Scholar
  7. 7.
    Detone, D., Malisiewicz, T., Rabinovich, A.: Toward geometric deep SLAM (2017)Google Scholar
  8. 8.
    ARKit Hardware and Software Integration. https://developer.apple.com/arkit/
  9. 9.
    Chekhlov, D., Gee, A.P., Calway, A., Mayol-Cuevas, W.: Ninja on a plane: automatic discovery of physical planes for augmented reality using visual SLAM. In: Proceedings of the 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 1–4. IEEE Computer Society (2007)Google Scholar
  10. 10.
    Klein, G., Murray, D.W.: Parallel tracking and mapping for small AR workspaces. In: international symposium on mixed and augmented reality, pp. 225–234 (2007)Google Scholar
  11. 11.
    Liu, H., Zhang, G., Bao, H.: Robust keyframe-based monocular SLAM for augmented reality. In: 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 1–10 (2016)Google Scholar
  12. 12.
    Xue, T., Luo, H., Cheng, D., Yuan, Z., Yang, X.: Real-time monocular dense mapping for augmented reality. In: Proceedings of the 2017 ACM on Multimedia Conference, pp. 510–518. ACM, Mountain View (2017)Google Scholar
  13. 13.
    Davison, A.J., Reid, I.D., Molton, N., Stasse, O.: MonoSLAM: real-time single camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29, 1052–1067 (2007)CrossRefGoogle Scholar
  14. 14.
    Murartal, R., Montiel, J.M.M., Tardos, J.D.: ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans. Rob. 31, 1147–1163 (2015)CrossRefGoogle Scholar
  15. 15.
    Concha, A., Civera, J.: DPPTAM: dense piecewise planar tracking and mapping from a monocular sequence. In: Intelligent Robots and Systems, pp. 5686–5693 (2015)Google Scholar
  16. 16.
    Gao, Q.H., Wan, T.R., Tang, W., Chen, L., Zhang, K.B.: An improved augmented reality registration method based on visual SLAM. In: Tian, F., Gatzidis, C., El Rhalibi, A., Tang, W., Charles, F. (eds.) Edutainment 2017. LNCS, vol. 10345, pp. 11–19. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-65849-0_2CrossRefGoogle Scholar
  17. 17.
    Lupton, T., Sukkarieh, S.: Removing scale biases and ambiguity from 6DoF monocular SLAM using inertial. In: International Conference on Robotics and Automation, pp. 3698–3703 (2008)Google Scholar
  18. 18.
    Kim, O., Kang, D.: A sensor fusion method to solve the scale ambiguity of single image by combining IMU. In: International Conference on Control and Automation, pp. 923–925 (2015)Google Scholar
  19. 19.
    Nutzi, G., Weiss, S., Scaramuzza, D., Siegwart, R.Y.: Fusion of IMU and vision for absolute scale estimation in monocular SLAM. J. Intell. Rob. Syst. 61, 287–299 (2011)CrossRefGoogle Scholar
  20. 20.
    Fujimoto, S., Hu, Z., Chapuis, R., Aufrere, R.: ORB-SLAM map initialization improvement using depth. In: International Conference on Image Processing, pp. 261–265 (2016)Google Scholar
  21. 21.
    Scaramuzza, D.: 1-point-RANSAC structure from motion for vehicle-mounted cameras by exploiting non-holonomic constraints. Int. J. Comput. Vis. 95, 74–85 (2011)CrossRefGoogle Scholar
  22. 22.
    Kitt, B., Rehder, J., Chambers, A., Schönbein, M., Lategahn, H., Singh, S.: Monocular visual odometry using a planar road model to solve scale ambiguity, pp. 43–48 (2011)Google Scholar
  23. 23.
    Song, S., Chandraker, M.: Robust scale estimation in real-time monocular SFM for autonomous driving. In: Computer Vision and Pattern Recognition, pp. 1566–1573 (2014)Google Scholar
  24. 24.
    Esteban, I., Dorst, L., Dijk, J.: Closed form solution for the scale ambiguity problem in monocular visual odometry. In: Liu, H., Ding, H., Xiong, Z., Zhu, X. (eds.) ICIRA 2010. LNCS (LNAI), vol. 6424, pp. 665–679. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-16584-9_64CrossRefGoogle Scholar
  25. 25.
    Li, Y., Wang, S., Yang, D., Sun, D.: Metric online monocular SLAM by using a known reference. In: World Congress on Intelligent Control and Automation, pp. 3002–3006 (2016)Google Scholar
  26. 26.
    Hartley, R.I., Zisserman, A.: Multiple view geometry in computer vision. Kybernetes 30, 1333–1341 (2000)zbMATHGoogle Scholar
  27. 27.
    Oberkampf, D., Dementhon, D., Davis, L.S.: Iterative pose estimation using coplanar feature points. Comput. Vis. Image Underst. 63, 495–511 (1996)CrossRefGoogle Scholar
  28. 28.
    Matas, J., Chum, O.: Randomized RANSAC with Td, d test. Image Vis. Comput. 22, 837–842 (2004)CrossRefGoogle Scholar
  29. 29.
    Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marín-Jiménez, M.J.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 47, 2280–2292 (2014)CrossRefGoogle Scholar
  30. 30.
    Holden, D., Komura, T., Saito, J.: Phase-functioned neural networks for character control. ACM Trans. Graph. 36, 1–13 (2017)CrossRefGoogle Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2018

Authors and Affiliations

  1. 1.Institute of AutomationChinese Academy of SciencesBeijingChina
  2. 2.School of Computer and Control EngineeringUniversity of Chinese Academy of SciencesBeijingChina
  3. 3.AICFVE of Beijing Film AcademyBeijingChina

Personalised recommendations