Skip to main content
Log in

An evaluation of real-time RGB-D visual odometry algorithms on mobile devices

  • Original Research Paper
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

We present an evaluation and a comparison of different visual odometry algorithms selected to be tested on a mobile device equipped with a RGB-D camera. The most promising algorithms from the literature are tested on different mobile devices, some equipped with the Structure Sensor. We evaluate the accuracy of each selected algorithm as well as the memory and CPU consumption, and we show that even without specific optimization, some of them can provide a reliable measure of the device motion in real time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

Notes

  1. We use the UNIX definition for CPU usage as \(\sum _{c \in NumCores} \dfrac{ time\_spent\_on\_core_c}{elapsed\_time}\), where \(elapsed\_time\) is the delta of system clock between the start and the end of the execution, and \(time\_spent\_on\_core_c\) are the number of system clocks spent on each of the NumCores of the machine [17].

  2. http://www.androidbenchmark.net/cpumark_chart.html.

  3. http://www.iphonebenchmark.net/cpumark_chart.html.

References

  1. Andreasson, H., Stoyanov, T.: Real time registration of RGB-D data using local visual features and 3D-NDT registration. In: Proceedings of International Conference on Robotics and Automation (ICRA) Workshop on Semantic Perception, Mapping and Exploration (SPME). IEEE Computer Society (2012)

  2. ARM. Arm compiler toolchain assembler reference v5.0—neon instructions. http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0489c/CJAJIIGG.html (2016)

  3. Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) Proceedings of the 2006 European Conference on Computer Vision (ECCV 2006), volume 3951 of Lecture Notes in Computer Science, pp. 404–417. Springer, Berlin (2006)

    Chapter  Google Scholar 

  4. Ben, A.: Principles of Concurrent and Distributed Programming, 2nd edn. Addison-Wesley, London (2006)

    Google Scholar 

  5. Besl, P.J., McKay, H.D.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)

    Article  Google Scholar 

  6. Borenstein, J., Everett, H.R., Feng, L.: Navigating Mobile Robots: Systems and Techniques. A. K. Peters Ltd, Natick, MA (1996)

    MATH  Google Scholar 

  7. Brunetto, N., Salti, S., Fioraio, N., Cavallari, T., Di Stefano, L.: Fusion of inertial and visual measurements for RGB-D SLAM on mobile devices. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops. IEEE Computer Society (2015)

  8. Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: Binary robust independent elementary features. In: Proceedings of the European Conference on Computer Vision (ECCV), volume 6314 LNCS, pp. 778–792. Springer, Berlin (2010)

    Chapter  Google Scholar 

  9. Chen, Y., Medioni, G.: Object modeling by registration of multiple range images. In: Proceedings of the 1991 IEEE International Conference on Robotics and Automation (ICRA), vol. 3, pp. 2724–2729 (1991)

  10. Driesen, K., Hölzle, U.: The direct cost of virtual function calls in C++. SIGPLAN Not. 31(10), 306–323 (1996)

    Article  Google Scholar 

  11. Dryanovski, I., Valenti, R.G., Xiao, J.: Fast visual odometry and mapping from RGB-D data. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2305–2310 (2013)

  12. Endres, F., Hess, J., Engelhard, N., Sturm, J., Cremers, D., Burgard, W.: An evaluation of the RGB-D SLAM system. In: Proceedings of the 2012 IEEE International Conference Robotics and Automation (ICRA), vol. 1, pp. 1691–1696. IEEE Computer Society (2012)

  13. Fang, Z., Zhang, Y.: Experimental evaluation of RGB-D visual odometry methods. Int. J. Adv. Robot. Syst. 12(3), 1–16 (2015)

    Article  Google Scholar 

  14. Geiger, A.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012), CVPR ’12, pp. 3354–3361. IEEE Computer Society, Washington, DC (2012)

  15. Geiger, A., Ziegler, J., Stiller, C.: StereoScan: Dense 3D reconstruction in real-time. In: Proceedings of the Intelligent Vehicles Symposium (IV 2011). IEEE Computer Society, Washington (2011)

  16. Google. ATAP Project Tango—Google. http://www.google.com/atap/projecttango/ (2014)

  17. Gunther, N.J.: White paper: UNIX Load Average—Part 1: How It Works. White paper http://www.teamquest.com/pdfs/whitepaper/ldavg1.pdf, TeamQuest Corporation (2010)

  18. Handa, A., Whelan, T., Mcdonald, J., Davison, A.J.: A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM. In: Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA 2014), pp. 1524–1531. IEEE Computer Society (2014)

  19. Horn, B.K.P.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. A 4(4), 629–642 (1987)

    Article  Google Scholar 

  20. Huang, A.S., Bachrach, A., Henry, P., Krainin, M., Maturana, D., Fox, D., Roy, N.: Visual odometry and mapping for autonomous flight using an RGB-D camera. In: Proceedings of the International Symposium of Robotics Research (ISRR 2011), pp. 1–16 (2011)

  21. Hudelist, M.A., Cobârzan, C., Schoeffmann, K.: OpenCV performance measurements on mobile devices. In: Proceedings of International Conference on Multimedia Retrieval—ICMR ’14, pp. 479–482. ACM Press, New York, NY (2014)

  22. Jones, G.: Accurate and computationally-inexpensive recovery of ego-motion using optical flow and range flow with extended temporal support. In: Proceedings of the British machine vision conference 2013, pp. 75.1–75.11. British Machine Vision Association (2013)

  23. Kerl, C., Sturm, J., Cremers, D.: Robust odometry estimation for RGB-D cameras. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2013), pp. 3748–3754. IEEE Computer Society (2013)

  24. Khoshelham, K., Elberink, S.O.: Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors (Basel, Switzerland) 12(2), 1437–54 (2012)

    Article  Google Scholar 

  25. Klingensmith, M., Dryanovski, I., Srinivasa, S., Xiao, J.: Chisel: Real time large scale 3d reconstruction onboard a mobile device. In: Proceedings of the 2015 Robotics Science and Systems Conference (2015)

  26. Kolev, K., Tanskanen, P., Speciale, P., Pollefeys, M.: Turning mobile phones into 3d scanners. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3946–3953. IEEE Computer Society (2014)

  27. Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: binary robust invariant scalable keypoints. In: Proceedings of the 2011 IEEE International Conference on Computer Vision (ICCV2011), pp. 2548–2555. IEEE Computer Society (2011)

  28. Limare, N.: Integer and floating-point arithmetic speed vs precision. http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/#index3h2 (2014)

  29. Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  30. Mantis Vision. MV4D depth sensor for mobile devices. http://www.mv4d.com/mobile.php (2014)

  31. Morell-Gimenez, V., Saval-Calvo, M., Azorin-Lopez, J., Garcia-Rodriguez, J., Cazorla, M., Orts-Escolano, S., Fuster-Guillo, A.: A comparative study of registration methods for RGB-D video of static scenes. Sensors (Switzerland) 14(5), 8547–8576 (2014)

    Article  Google Scholar 

  32. Newcombe, R.A., Molyneaux, D., Kim, D., Davison, A.J., Shotton, J., Hodges, S., Fitzgibbon, A.: KinectFusion: real-time dense surface mapping and tracking. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2011), pp. 127–136. IEEE Computer Society (2011)

  33. Nguyen, C.V., Izadi, S., Lovell, D.: Modeling kinect sensor noise for improved 3d reconstruction and tracking. In: Proceedings of the 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission, 3DIMPVT ’12, pp. 524–530. IEEE Computer Society, Washington, DC (2012)

  34. Nister, D., Naroditsky, O., Bergen, J.: Visual odometry. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), vol. 1, pp. I–652–I–659. IEEE Computer Society (2004)

  35. Occipital. ST depth. STTracker instance with the kSTTrackerTypeKey set to STTrackerDepthBased (2013)

  36. Occipital. ST hybrid. STTracker instance with the kSTTrackerTypeKey set to STTrackerDepthAndColorBased (2013)

  37. Occipital Inc. The structure sensor. http://structure.io (2014)

  38. Ondrúška, P., Kohli, P., Izadi, S.: Mobilefusion: real-time volumetric surface reconstruction and dense tracking on mobile phones. IEEE Trans. Vis. Comput. Graph. 21(11), 1251–1258 (2015)

    Article  Google Scholar 

  39. Pomerleau, F., Colas, F., Siegwart, R.: A review of point cloud registration algorithms for mobile robotics. Found. Trends Robot. 4(1–104), 1–104 (2015)

    Google Scholar 

  40. Prisacariu, V., Kähler, O., Murray, D., Reid, I.: Simultaneous 3d tracking and reconstruction on a mobile phone. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2013), pp. 89–98. IEEE Computer Society (2013)

  41. Prisacariu, V.A., Kähler, Murray: Real-time 3d tracking and reconstruction on mobile phones. IEEE Trans. Vis. Comput. Graph. 21(5), 557–570x (2012)

    Article  Google Scholar 

  42. Quigley, M., Conley, K., Gerkey, B.P., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source robot operating system. In: ICRA Workshop on Open Source Software (2009)

  43. Rabaud, V., Dimashova, M.: OpenCV ICP and RGBD visual odometry. Class RgbdICPOdometry and function RGBDICPOdometryImpl() in the file https://github.com/Itseez/opencv_contrib/tree/master/modules/rgbd/src/odometry.cpp (2012)

  44. Rabaud, V., Dimashova, M.: OpenCV ICP-based visual odometry. Class ICPOdometry and function RGBDICPOdometryImpl() in the file https://github.com/Itseez/opencv_contrib/tree/master/modules/rgbd/src/odometry.cpp (2012)

  45. Rabaud, V., Dimashova, M.: OpenCV RGBD-based visual odometry. Class RgbdOdometry and function RGBDICPOdometryImpl() in the file https://github.com/Itseez/opencv_contrib/tree/master/modules/rgbd/src/odometry.cpp (2012)

  46. Rabaud, V., Dimashova, M.: OpenCV RGBD module. https://github.com/Itseez/opencv_contrib/tree/master/modules/rgbd (2012)

  47. Raynal, M.: Concurrent Programming: Algorithms, Principles, and Foundations. Springer, Berlin (2012)

    MATH  Google Scholar 

  48. Rosten, E., Porter, R., Drummond, T.: Faster and better: a machine learning approach to corner detection. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 105–119 (2010)

    Article  Google Scholar 

  49. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: An efficient alternative to SIFT or SURF. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2564–2571. IEEE Computer Society (2011)

  50. Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings of the Third International Conference on 3-D Digital Imaging and Modeling, pp. 145–152. IEEE (2001)

  51. Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China. IEEE Computer Society, May 9–13 (2011)

  52. Scaramuzza, D., Fraundorfer, F.: Visual odometry [tutorial]. IEEE Robot. Autom. Mag. 18(4), 80–92 (2011)

    Article  Google Scholar 

  53. Schöps, T., Engel, J., Cremers, D.: Semi-dense visual odometry for AR on a smartphone. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2014). IEEE Computer Society (2014)

  54. Schöps, T., Sattler, T., Häne, C., Pollefeys, M.: 3D modeling on the go: interactive 3D reconstruction of large-scale scenes on mobile devices. In: Proceedings of the International Conference on 3D Vision, Piscataway, NJ. IEEE Computer Society (2015)

  55. Steinbrücker, F., Sturm, J., Cremers, D.: Real-time visual odometry from dense RGB-D images. In: Proceedings of the IEEE International Conference on Computer Vision, number 3, pp. 719–722. IEEE Computer Society (2011)

  56. Stückler, J., Behnke, S.: Multi-resolution surfel maps for efficient dense 3D modeling and tracking. J. Vis. Commun. Image Represent. 25(1), 137–147 (2014)

    Article  Google Scholar 

  57. Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS 2012), pp. 573–580. IEEE Computer Society (2012)

  58. Tanskanen, P., Kolev, K., Meier, L., Camposeco, F., Saurer, O., Pollefeys, M.: Live metric 3d reconstruction on mobile phones. In: Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV), pp. 65–72. IEEE Computer Society (2013)

  59. Tomasi, C., Manduchi, R.: Bilateral Filtering for Gray and Color Images. In: Proceedings of the 1998 IEEE International Conference on Computer Vision (ICCV), pp. 839–846. IEEE Computer Society (1998)

  60. Whelan, T., Johannsson, H., Kaess, M., Leonard, J., McDonald, J.: Robust real-time visual odometry for dense RGB-D mapping. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2013), Karlsruhe, Germany. IEEE Computer Society (2013)

  61. Yang, D.: C++ and Object-Oriented Numeric Computing for Scientists and Engineers. Springer, New York (2001)

    Book  Google Scholar 

  62. Yousif, K., Bab-Hadiashar, A., Hoseinnezhad, R.: An overview to visual odometry and visual slam: applications to mobile robotics. Intell. Ind. Syst. 1(4), 289–311 (2015)

    Article  Google Scholar 

  63. Zhang, J., Kaess, M., Singh, S.: Real-time depth enhanced monocular odometry. In: Proceedings of the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4973–4980. IEEE Computer Society (2014)

  64. Zhang, J., Kaess, M., Singh, S.: A real-time method for depth enhanced visual odometry. Auton. Robots 41(1), 31–43 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

This research was supported by the CIFRE grant ANRT contract #2014/0014 funding the collaboration between the French company Telequid and INPT-IRIT laboratory, and in part by the Croatian Science Foundation’s funding of the project IP-11-2013-3717. The authors also acknowledge the French-Croatian Program “Cogito”, Hubert Curien partnership, funding the project “3D reconstruction using smartphone” (project code 33059RF).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Vincent Angladon.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Angladon, V., Gasparini, S., Charvillat, V. et al. An evaluation of real-time RGB-D visual odometry algorithms on mobile devices. J Real-Time Image Proc 16, 1643–1660 (2019). https://doi.org/10.1007/s11554-017-0670-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11554-017-0670-y

Keywords

Navigation