Skip to main content
SpringerLink
Log in

A Tracking Registration Method for Augmented Reality Based on Multi-modal Template Matching and Point Clouds

  • Research Article
  • Published:
International Journal of Automation and Computing Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

In order to overcome the defects where the surface of the object lacks sufficient texture features and the algorithm cannot meet the real-time requirements of augmented reality, a markerless augmented reality tracking registration method based on multimodal template matching and point clouds is proposed. The method first adapts the linear parallel multi-modal LineMod template matching method with scale invariance to identify the texture-less target and obtain the reference image as the key frame that is most similar to the current perspective. Then, we can obtain the initial pose of the camera and solve the problem of re-initialization because of tracking registration interruption. A point cloud-based method is used to calculate the precise pose of the camera in real time. In order to solve the problem that the traditional iterative closest point (ICP) algorithm cannot meet the real-time requirements of the system, Kd-tree (k-dimensional tree) is used under the graphics processing unit (GPU) to replace the part of finding the nearest points in the original ICP algorithm to improve the speed of tracking registration. At the same time, the random sample consensus (RANSAC) algorithm is used to remove the error point pairs to improve the accuracy of the algorithm. The results show that the proposed tracking registration method has good real-time performance and robustness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. P. X. Cao, W. X. Li, W. P. Ma. Tracking registration method based on improved random ferns. Computer Applications and Software, vol. 36, no. 11, pp. 51–56, 2019. DOI: https://doi.org/10.3969/j.issn.1000-386x.2019.11.009. (in Chinese)

    Google Scholar 

  2. Q. B. Tang, S. M. Hou. Research on tracking and registration based on improved ORB algorithm and pose estimation. Application Research of Computers, vol. 33, no. 12, pp. 3905–3908, 2016. DOI: https://doi.org/10.3969/j.issn.1001-3695.2016.12.086. (in Chinese)

    Google Scholar 

  3. S. M. Hou, J. Han, Y. D. Zhang, Z. Q. Zhu. Survey of vision-based augmented reality 3D registration technology. Journal of System Simulation, vol. 31, no. 11, pp. 2206–2215, 2019. DOI: https://doi.org/10.16182/j.issn1004731x.joss.19-FZ0286. (in Chinese)

    Google Scholar 

  4. J. Chen, Y. Sun. System algorithm based on FAST keypoints for markerless augmented reality applications. Transactions of Beijing Institute of Technology, vol. 35, no. 4, pp. 421–426, 2015. DOI: https://doi.org/10.15918/j.tbit1001-0645.2015.04.017. (in Chinese)

    Google Scholar 

  5. Y. Dong, L. Ji, S. Wang, P. Gong, J. Yue, R. Shen, C. Chen, Y. Zhang. Accurate 6DOF Pose Tracking for Texture-Less Objects. IEEE Transactions on Circuits and Systems for Video Technology, published online, 2020. DOI: https://doi.org/10.1109/TCSVT.2020.3011737.

  6. P. X. Cao, W. X. Li, W. P. Ma. Tracking registration algorithm for augmented reality based on template tracking. International Journal of Automation and Computing, vol. 17, no. 2, pp. 257–266, 2020. DOI: https://doi.org/10.1007/s11633-019-1198-3.

    Article  Google Scholar 

  7. N. Payet, S. Todorovic. From contours to 3D object detection and pose estimation. In Proceedings of International Conference on Computer Vision, IEEE, Barcelona, Spain, pp. 983–990, 2011. DOI: https://doi.org/10.1109/ICCV.2011.6126342.

    Google Scholar 

  8. L. H. Pan, F. Q. Tian, W. J. Ying, Q. J. Qian. Survey on direct-method visual simultaneous localization and mapping. Application Research of Computers, vol. 36, no. 4, pp. 961–966, 2019. DOI: https://doi.org/10.19734/j.issn.1001-3695.2018.01.0123. (in Chinese)

    Google Scholar 

  9. L. Chen, W. Tang, N. W. John, T. R. Wan, J. J. Zhang. SLAM-based dense surface reconstruction in monocular Minimally Invasive Surgery and its application to Augmented Reality. Computer Methods and Programs in Biomedicine, vol. 158, pp. 135–146, 2018. DOI: https://doi.org/10.1016/j.cmpb.2018.02.006.

    Article  Google Scholar 

  10. Y. Wang, S. S. Zhang, W. P. He, X. L. Bai. Model-based marker-less 3D tracking approach for augmented reality. Journal of Shanghai Jiaotong University, vol. 52, no. 1, pp. 83–89, 2018. DOI: https://doi.org/10.16183/j.cnki.jsjtu.2018.01.013. (in Chinese)

    Google Scholar 

  11. Y. Park, V. Lepetit, W. Woo. Texture-less object tracking with online training using an RGB-D camera. In Proceedings of the 10th IEEE International Symposium on Mixed and Augmented Reality, IEEE, Basel, Switzerland, pp. 121–126, 2011. DOI: https://doi.org/10.1109/ISMAR.2011.6092377.

    Google Scholar 

  12. Y. Wang, S. S. Zhang, X. L. Bai. A 3D tracking and registration method based on point cloud and visual features for augmented reality aided assembly system. Journal of Northwestern Polytechnical University, vol. 37, no. 1, pp. 143–151, 2019. DOI: https://doi.org/10.3969/j.issn.1000-2758.2019.01.021. (in Chinese)

    Article  Google Scholar 

  13. S. Hinterstoisser, S. Holzer, C. Cagniart, S. Ilic, K. Konolige, N. Navab, V. Lepetit. Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenes. In Proceedings of International Conference on Computer Vision, IEEE, Barcelona, Spain, pp. 858–865, 2011. DOI: https://doi.org/10.1109/ICCV.2011.6126326.

    Google Scholar 

  14. C. Tomasi, R. Manduchi. Bilateral filtering for gray and color images. In Proceedings of the 6th International Conference on Computer Vision, IEEE, Bombay, India, pp. 839–846, 1998. DOI: https://doi.org/10.1109/ICCV.1998.710815.

    Google Scholar 

  15. Y. Wang, S. S. Zhang, S. Yang, W. P. He, X. L. Bai, Y. F. Zeng. A LINE-MOD-based markerless tracking approach-for AR applications. The International Journal of Advanced Manufacturing Technology, vol. 89, no. 5–8, pp. 1699–1707, 2017. DOI: https://doi.org/10.1007/s00170-016-9180-5.

    Article  Google Scholar 

  16. S. Hinterstoisser, C. Cagniart, S. Ilic, P. Sturm, N. Navab, P. Fua, V. Lepetit. Gradient response maps for real-time detection of textureless objects. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 5, pp. 876–888, 2012. DOI: https://doi.org/10.1109/TPAMI.2011.206.

    Article  Google Scholar 

  17. T. Ji, F. D. Li, Y. Q. Yang, T. P. Zhang. Monocular vision pose measurement based on improved ICP algorithm and CAD model. Journal of Yangzhou University (Natural Science Edition), vol. 22, no. 1, pp. 50–54, 61, 2019. DOI: https://doi.org/10.19411/j.1007-824x.2019.01.011. (in Chinese)

  18. K. S. Wang, X. Li, H. W. Lei, X. R. Zhang. An ICP algorithm based on block path closest point search. Journal of Physics: Conference Series, vol. 887, Article number 012063, 2017. DOI: https://doi.org/10.1088/1742-6596/887/1/012063.

  19. W. Guan, W. T. Li, Y. Ren. Point cloud registration based on improved ICP algorithm. In Proceedings of Chinese Control and Decision Conference, IEEE, Shenyang, China, pp. 1461–1465, 2018. DOI: https://doi.org/10.1109/CCDC.2018.8407357.

    Google Scholar 

  20. H. Yu, H. J. Du, Y. H. Cai. Object point cloud modeling method based on improved SIFT-ICP algorithm. High Technology Letters, vol. 29, no. 344, pp. 24–31, 2019.

    Google Scholar 

  21. J. Liu, J. W. Zhu, J. L. Yang, X. L. Meng, H. Zhang. Three-dimensional point cloud registration based on ICP algorithm employing K-D tree optimization. In Proceedings of the 8th International Conference on Digital Image Processing, Chengdu, China, Article number 100334D, 2016. DOI: https://doi.org/10.1117/12.2248362.

  22. L. J. Hu, S. Nooshabadi. Massive parallelization of approximate nearest neighbor search on Kd-tree for high-dimensional image descriptor matching. Journal of Visual Communication and Image Representation, vol. 44, pp. 106–115, 2017. DOI: https://doi.org/10.1016/j.jvcir.2017.01.013.

    Article  Google Scholar 

  23. M. Zhang. Study of Kd-tree based GPU ray tracing, Master dissertation, Chang’an University, China, 2014. DOI: https://doi.org/10.7666/d.D558268. (in Chinese)

    Google Scholar 

  24. M. D. Li, S. P. Jiang, H. P. Wang. A RANSAC-based stable plane fitting method of point clouds. Science of Surveying and Mapping, vol. 40, no. 1, pp. 102–106, 2015. DOI: https://doi.org/10.16251/j.cnki.1009-2307.2015.01.022. (in Chinese)

    Google Scholar 

  25. E. Arias-Castro, J. Wang. RANSAC algorithms for subspace recovery and subspace clustering. [Online], Available: https://arxiv.org/abs/1711.11220, 2017.

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (No. 61125101).

Author information

Authors and Affiliations

  1. Lanzhou Institute of Physics, China Academy of Space Technology, Lanzhou, 730000, China

    Peng-Xia Cao, Wen-Xin Li & Wei-Ping Ma

Authors
  1. Peng-Xia Cao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen-Xin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei-Ping Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peng-Xia Cao.

Additional information

Colored figures are available in the online version at https://link.springer.com/journal/11633

Peng-Xia Cao received the B. Eng. degree in communication engineering from Hunan International Economics University, China in 2011, and the M. Eng. degree in circuits and systems from Hunan Normal University, China in 2015. Currently, she is a Ph. D. degree candidate in space electronics at Lanzhou Institute of Physics, China Academy of Space Technology (CAST), China.

Her research interests include space electronic technology, computer vision and augmented reality.

Wen-Xin Li received the M. Eng. degree in applied mathematics from Northwestern Polytechnical University, China in 1993, and the Ph. D. degree in automatic control from Northwestern Polytechnical University, China in 2011. Currently, he is a researcher at Lanzhou Institute of Physics, CAST, China.

His research interests include space electronic technology, software reuse technology, system simulation and reconstruction technology.

Wei-Ping Ma received the B. Eng. and M.Eng. degrees in electronic information science and technology from Xi’an University of Science and technology, China in 2011 and 2015, respectively. Currently, she is a Ph. D. degree candidate in space electronics at Lanzhou Institute of Physics, CAST, China.

Her research interests include space electronic technology, computer vision and intelligent robotics.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, PX., Li, WX. & Ma, WP. A Tracking Registration Method for Augmented Reality Based on Multi-modal Template Matching and Point Clouds. Int. J. Autom. Comput. 18, 288–299 (2021). https://doi.org/10.1007/s11633-020-1265-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-020-1265-9

Keywords

Search

Navigation