Abstract
We present an evaluation and a comparison of different visual odometry algorithms selected to be tested on a mobile device equipped with a RGB-D camera. The most promising algorithms from the literature are tested on different mobile devices, some equipped with the Structure Sensor. We evaluate the accuracy of each selected algorithm as well as the memory and CPU consumption, and we show that even without specific optimization, some of them can provide a reliable measure of the device motion in real time.
Similar content being viewed by others
Notes
We use the UNIX definition for CPU usage as \(\sum _{c \in NumCores} \dfrac{ time\_spent\_on\_core_c}{elapsed\_time}\), where \(elapsed\_time\) is the delta of system clock between the start and the end of the execution, and \(time\_spent\_on\_core_c\) are the number of system clocks spent on each of the NumCores of the machine [17].
References
Andreasson, H., Stoyanov, T.: Real time registration of RGB-D data using local visual features and 3D-NDT registration. In: Proceedings of International Conference on Robotics and Automation (ICRA) Workshop on Semantic Perception, Mapping and Exploration (SPME). IEEE Computer Society (2012)
ARM. Arm compiler toolchain assembler reference v5.0—neon instructions. http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0489c/CJAJIIGG.html (2016)
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: Speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) Proceedings of the 2006 European Conference on Computer Vision (ECCV 2006), volume 3951 of Lecture Notes in Computer Science, pp. 404–417. Springer, Berlin (2006)
Ben, A.: Principles of Concurrent and Distributed Programming, 2nd edn. Addison-Wesley, London (2006)
Besl, P.J., McKay, H.D.: A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)
Borenstein, J., Everett, H.R., Feng, L.: Navigating Mobile Robots: Systems and Techniques. A. K. Peters Ltd, Natick, MA (1996)
Brunetto, N., Salti, S., Fioraio, N., Cavallari, T., Di Stefano, L.: Fusion of inertial and visual measurements for RGB-D SLAM on mobile devices. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV) Workshops. IEEE Computer Society (2015)
Calonder, M., Lepetit, V., Strecha, C., Fua, P.: BRIEF: Binary robust independent elementary features. In: Proceedings of the European Conference on Computer Vision (ECCV), volume 6314 LNCS, pp. 778–792. Springer, Berlin (2010)
Chen, Y., Medioni, G.: Object modeling by registration of multiple range images. In: Proceedings of the 1991 IEEE International Conference on Robotics and Automation (ICRA), vol. 3, pp. 2724–2729 (1991)
Driesen, K., Hölzle, U.: The direct cost of virtual function calls in C++. SIGPLAN Not. 31(10), 306–323 (1996)
Dryanovski, I., Valenti, R.G., Xiao, J.: Fast visual odometry and mapping from RGB-D data. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), pp. 2305–2310 (2013)
Endres, F., Hess, J., Engelhard, N., Sturm, J., Cremers, D., Burgard, W.: An evaluation of the RGB-D SLAM system. In: Proceedings of the 2012 IEEE International Conference Robotics and Automation (ICRA), vol. 1, pp. 1691–1696. IEEE Computer Society (2012)
Fang, Z., Zhang, Y.: Experimental evaluation of RGB-D visual odometry methods. Int. J. Adv. Robot. Syst. 12(3), 1–16 (2015)
Geiger, A.: Are we ready for autonomous driving? The KITTI vision benchmark suite. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2012), CVPR ’12, pp. 3354–3361. IEEE Computer Society, Washington, DC (2012)
Geiger, A., Ziegler, J., Stiller, C.: StereoScan: Dense 3D reconstruction in real-time. In: Proceedings of the Intelligent Vehicles Symposium (IV 2011). IEEE Computer Society, Washington (2011)
Google. ATAP Project Tango—Google. http://www.google.com/atap/projecttango/ (2014)
Gunther, N.J.: White paper: UNIX Load Average—Part 1: How It Works. White paper http://www.teamquest.com/pdfs/whitepaper/ldavg1.pdf, TeamQuest Corporation (2010)
Handa, A., Whelan, T., Mcdonald, J., Davison, A.J.: A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM. In: Proceedings of the 2014 IEEE International Conference on Robotics and Automation (ICRA 2014), pp. 1524–1531. IEEE Computer Society (2014)
Horn, B.K.P.: Closed-form solution of absolute orientation using unit quaternions. J. Opt. Soc. Am. A 4(4), 629–642 (1987)
Huang, A.S., Bachrach, A., Henry, P., Krainin, M., Maturana, D., Fox, D., Roy, N.: Visual odometry and mapping for autonomous flight using an RGB-D camera. In: Proceedings of the International Symposium of Robotics Research (ISRR 2011), pp. 1–16 (2011)
Hudelist, M.A., Cobârzan, C., Schoeffmann, K.: OpenCV performance measurements on mobile devices. In: Proceedings of International Conference on Multimedia Retrieval—ICMR ’14, pp. 479–482. ACM Press, New York, NY (2014)
Jones, G.: Accurate and computationally-inexpensive recovery of ego-motion using optical flow and range flow with extended temporal support. In: Proceedings of the British machine vision conference 2013, pp. 75.1–75.11. British Machine Vision Association (2013)
Kerl, C., Sturm, J., Cremers, D.: Robust odometry estimation for RGB-D cameras. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2013), pp. 3748–3754. IEEE Computer Society (2013)
Khoshelham, K., Elberink, S.O.: Accuracy and resolution of Kinect depth data for indoor mapping applications. Sensors (Basel, Switzerland) 12(2), 1437–54 (2012)
Klingensmith, M., Dryanovski, I., Srinivasa, S., Xiao, J.: Chisel: Real time large scale 3d reconstruction onboard a mobile device. In: Proceedings of the 2015 Robotics Science and Systems Conference (2015)
Kolev, K., Tanskanen, P., Speciale, P., Pollefeys, M.: Turning mobile phones into 3d scanners. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3946–3953. IEEE Computer Society (2014)
Leutenegger, S., Chli, M., Siegwart, R.Y.: BRISK: binary robust invariant scalable keypoints. In: Proceedings of the 2011 IEEE International Conference on Computer Vision (ICCV2011), pp. 2548–2555. IEEE Computer Society (2011)
Limare, N.: Integer and floating-point arithmetic speed vs precision. http://nicolas.limare.net/pro/notes/2014/12/12_arit_speed/#index3h2 (2014)
Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Mantis Vision. MV4D depth sensor for mobile devices. http://www.mv4d.com/mobile.php (2014)
Morell-Gimenez, V., Saval-Calvo, M., Azorin-Lopez, J., Garcia-Rodriguez, J., Cazorla, M., Orts-Escolano, S., Fuster-Guillo, A.: A comparative study of registration methods for RGB-D video of static scenes. Sensors (Switzerland) 14(5), 8547–8576 (2014)
Newcombe, R.A., Molyneaux, D., Kim, D., Davison, A.J., Shotton, J., Hodges, S., Fitzgibbon, A.: KinectFusion: real-time dense surface mapping and tracking. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2011), pp. 127–136. IEEE Computer Society (2011)
Nguyen, C.V., Izadi, S., Lovell, D.: Modeling kinect sensor noise for improved 3d reconstruction and tracking. In: Proceedings of the 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission, 3DIMPVT ’12, pp. 524–530. IEEE Computer Society, Washington, DC (2012)
Nister, D., Naroditsky, O., Bergen, J.: Visual odometry. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2004), vol. 1, pp. I–652–I–659. IEEE Computer Society (2004)
Occipital. ST depth. STTracker instance with the kSTTrackerTypeKey set to STTrackerDepthBased (2013)
Occipital. ST hybrid. STTracker instance with the kSTTrackerTypeKey set to STTrackerDepthAndColorBased (2013)
Occipital Inc. The structure sensor. http://structure.io (2014)
Ondrúška, P., Kohli, P., Izadi, S.: Mobilefusion: real-time volumetric surface reconstruction and dense tracking on mobile phones. IEEE Trans. Vis. Comput. Graph. 21(11), 1251–1258 (2015)
Pomerleau, F., Colas, F., Siegwart, R.: A review of point cloud registration algorithms for mobile robotics. Found. Trends Robot. 4(1–104), 1–104 (2015)
Prisacariu, V., Kähler, O., Murray, D., Reid, I.: Simultaneous 3d tracking and reconstruction on a mobile phone. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2013), pp. 89–98. IEEE Computer Society (2013)
Prisacariu, V.A., Kähler, Murray: Real-time 3d tracking and reconstruction on mobile phones. IEEE Trans. Vis. Comput. Graph. 21(5), 557–570x (2012)
Quigley, M., Conley, K., Gerkey, B.P., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source robot operating system. In: ICRA Workshop on Open Source Software (2009)
Rabaud, V., Dimashova, M.: OpenCV ICP and RGBD visual odometry. Class RgbdICPOdometry and function RGBDICPOdometryImpl() in the file https://github.com/Itseez/opencv_contrib/tree/master/modules/rgbd/src/odometry.cpp (2012)
Rabaud, V., Dimashova, M.: OpenCV ICP-based visual odometry. Class ICPOdometry and function RGBDICPOdometryImpl() in the file https://github.com/Itseez/opencv_contrib/tree/master/modules/rgbd/src/odometry.cpp (2012)
Rabaud, V., Dimashova, M.: OpenCV RGBD-based visual odometry. Class RgbdOdometry and function RGBDICPOdometryImpl() in the file https://github.com/Itseez/opencv_contrib/tree/master/modules/rgbd/src/odometry.cpp (2012)
Rabaud, V., Dimashova, M.: OpenCV RGBD module. https://github.com/Itseez/opencv_contrib/tree/master/modules/rgbd (2012)
Raynal, M.: Concurrent Programming: Algorithms, Principles, and Foundations. Springer, Berlin (2012)
Rosten, E., Porter, R., Drummond, T.: Faster and better: a machine learning approach to corner detection. IEEE Trans. Pattern Anal. Mach. Intell. 32(1), 105–119 (2010)
Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: An efficient alternative to SIFT or SURF. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2564–2571. IEEE Computer Society (2011)
Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings of the Third International Conference on 3-D Digital Imaging and Modeling, pp. 145–152. IEEE (2001)
Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: Proceedings of the 2011 IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China. IEEE Computer Society, May 9–13 (2011)
Scaramuzza, D., Fraundorfer, F.: Visual odometry [tutorial]. IEEE Robot. Autom. Mag. 18(4), 80–92 (2011)
Schöps, T., Engel, J., Cremers, D.: Semi-dense visual odometry for AR on a smartphone. In: Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR 2014). IEEE Computer Society (2014)
Schöps, T., Sattler, T., Häne, C., Pollefeys, M.: 3D modeling on the go: interactive 3D reconstruction of large-scale scenes on mobile devices. In: Proceedings of the International Conference on 3D Vision, Piscataway, NJ. IEEE Computer Society (2015)
Steinbrücker, F., Sturm, J., Cremers, D.: Real-time visual odometry from dense RGB-D images. In: Proceedings of the IEEE International Conference on Computer Vision, number 3, pp. 719–722. IEEE Computer Society (2011)
Stückler, J., Behnke, S.: Multi-resolution surfel maps for efficient dense 3D modeling and tracking. J. Vis. Commun. Image Represent. 25(1), 137–147 (2014)
Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: Proceedings of the IEEE International Conference on Intelligent Robots and Systems (IROS 2012), pp. 573–580. IEEE Computer Society (2012)
Tanskanen, P., Kolev, K., Meier, L., Camposeco, F., Saurer, O., Pollefeys, M.: Live metric 3d reconstruction on mobile phones. In: Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV), pp. 65–72. IEEE Computer Society (2013)
Tomasi, C., Manduchi, R.: Bilateral Filtering for Gray and Color Images. In: Proceedings of the 1998 IEEE International Conference on Computer Vision (ICCV), pp. 839–846. IEEE Computer Society (1998)
Whelan, T., Johannsson, H., Kaess, M., Leonard, J., McDonald, J.: Robust real-time visual odometry for dense RGB-D mapping. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA 2013), Karlsruhe, Germany. IEEE Computer Society (2013)
Yang, D.: C++ and Object-Oriented Numeric Computing for Scientists and Engineers. Springer, New York (2001)
Yousif, K., Bab-Hadiashar, A., Hoseinnezhad, R.: An overview to visual odometry and visual slam: applications to mobile robotics. Intell. Ind. Syst. 1(4), 289–311 (2015)
Zhang, J., Kaess, M., Singh, S.: Real-time depth enhanced monocular odometry. In: Proceedings of the 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4973–4980. IEEE Computer Society (2014)
Zhang, J., Kaess, M., Singh, S.: A real-time method for depth enhanced visual odometry. Auton. Robots 41(1), 31–43 (2015)
Acknowledgements
This research was supported by the CIFRE grant ANRT contract #2014/0014 funding the collaboration between the French company Telequid and INPT-IRIT laboratory, and in part by the Croatian Science Foundation’s funding of the project IP-11-2013-3717. The authors also acknowledge the French-Croatian Program “Cogito”, Hubert Curien partnership, funding the project “3D reconstruction using smartphone” (project code 33059RF).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Angladon, V., Gasparini, S., Charvillat, V. et al. An evaluation of real-time RGB-D visual odometry algorithms on mobile devices. J Real-Time Image Proc 16, 1643–1660 (2019). https://doi.org/10.1007/s11554-017-0670-y
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11554-017-0670-y