Advertisement

Improved Structure from Motion Using Fiducial Marker Matching

  • Joseph DeGolEmail author
  • Timothy Bretl
  • Derek Hoiem
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11207)

Abstract

In this paper, we present an incremental structure from motion (SfM) algorithm that significantly outperforms existing algorithms when fiducial markers are present in the scene, and that matches the performance of existing algorithms when no markers are present. Our algorithm uses markers to limit potential incorrect image matches, change the order in which images are added to the reconstruction, and enforce new bundle adjustment constraints. To validate our algorithm, we introduce a new dataset with 16 image collections of large indoor scenes with challenging characteristics (e.g., blank hallways, glass facades, brick walls) and with markers placed throughout. We show that our algorithm produces complete, accurate reconstructions on all 16 image collections, most of which cause other algorithms to fail. Further, by selectively masking fiducial markers, we show that the presence of even a small number of markers can improve the results of our algorithm.

Keywords

Structure from motion SFM Fiducial markers 3D reconstruction Simultaneous localization and mapping SLAM 

Notes

Acknowledgement

This work is supported by NSF Grant CMMI-1446765 and the DoD National Defense Science and Engineering Graduate Fellowship (NDSEG). Thank you also to Reconstruct for computational resources that enabled this research and Daniel Yuan, Jae Yong Lee, and Shreya Jagarlamudi for help with data collection.

Supplementary material

474178_1_En_17_MOESM1_ESM.pdf (10.5 mb)
Supplementary material 1 (pdf 10797 KB)

References

  1. 1.
    Birdal, T., Dobryden, I., Ilic, S.: X-tag: a fiducial tag for flexible and accurate bundle adjustment. In: 2016 Fourth International Conference on 3D Vision (3DV), pp. 556–564, October 2016Google Scholar
  2. 2.
    Wang, J., Olson, E.: AprilTag 2: efficient and robust fiducial detection. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), October 2016Google Scholar
  3. 3.
    DeGol, J., Bretl, T., Hoiem, D.: ChromaTag: a colored marker and fast detection algorithm. In: ICCV (2017)Google Scholar
  4. 4.
    Garrido-Jurado, S., noz Salinas, R.M., Madrid-Cuevas, F., Marín-Jiménez, M.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 47(6), 2280–2292 (2014)CrossRefGoogle Scholar
  5. 5.
    Fiala, M.: Designing highly reliable fiducial markers. IEEE Trans. Pattern Anal. Mach. Intell. 32(7), 1317–1324 (2010)CrossRefGoogle Scholar
  6. 6.
    Bergamasco, F., Albarelli, A., Cosmo, L., Rodola, E., Torsello, A.: An accurate and robust artificial marker based on cyclic codes. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2016)Google Scholar
  7. 7.
    Calvet, L., Gurdjos, P., Griwodz, C., Gasparini, S.: Detection and accurate localization of circular fiducials under highly challenging conditions. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016Google Scholar
  8. 8.
  9. 9.
    Muoz-Salinas, R., Marn-Jimenez, M.J., Yeguas-Bolivar, E., Medina-Carnicer, R.: Mapping and localization from planar markers. Pattern Recogn. 73, 158–171 (2018)CrossRefGoogle Scholar
  10. 10.
    Schaffalitzky, F., Zisserman, A.: Multi-view matching for unordered image sets, or “How Do I Organize My Holiday Snaps?”. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002. LNCS, vol. 2350, pp. 414–431. Springer, Heidelberg (2002).  https://doi.org/10.1007/3-540-47969-4_28CrossRefGoogle Scholar
  11. 11.
    Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in 3D. In: Proceedings of ACM SIGGRAPH (2006)Google Scholar
  12. 12.
    Agarwal, S., Snavely, N., Simon, I., Seitz, S.M., Szeliski, R.: Building Rome in a day. In: IEEE 12th International Conference on Computer Vision, pp. 72–79, September 2009Google Scholar
  13. 13.
    Frahm, J.-M., et al.: Building Rome on a cloudless day. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6314, pp. 368–381. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15561-1_27CrossRefGoogle Scholar
  14. 14.
    Wu, C.: Towards linear-time incremental structure from motion. In: 2013 International Conference on 3D Vision - 3DV 2013 (2013)Google Scholar
  15. 15.
    Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  16. 16.
    Neunert, M., Bloesch, M., Buchli, J.: An open source, fiducial based, visual-inertial motion capture system. In: 2016 19th International Conference on Information Fusion (FUSION) (2016)Google Scholar
  17. 17.
    Klopschitz, M., Schmalstieg, D.: Automatic reconstruction of wide-area fiducial marker models. In: 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality (2007)Google Scholar
  18. 18.
    Lim, H., Lee, Y.S.: Real-time single camera slam using fiducial markers. In: 2009 ICCAS-SICE (2009)Google Scholar
  19. 19.
    Yamada, T., Yairi, T., Bener, S.H., Machida, K.: A study on slam for indoor blimp with visual markers. In: ICCAS-SICE 2009, pp. 647–652 (2009)Google Scholar
  20. 20.
    Feng, C., Kamat, V., Menassa, C.C.: Marker-assisted structure from motion for 3D environment modeling and object pose estimation. In: Construction Research Congress (2016)Google Scholar
  21. 21.
    Schweighofer, G., Pinz, A.: Robust pose estimation from a planar target. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2024–2030 (2006)CrossRefGoogle Scholar
  22. 22.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)MathSciNetCrossRefGoogle Scholar
  23. 23.
    Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004). ISBN 0521540518CrossRefGoogle Scholar
  24. 24.
    Moulon, P., Monasse, P., Marlet, R., et al.: OpenMVG. An open multiple view geometry library. https://github.com/openMVG/openMVG
  25. 25.
    Moulon, P., Monasse, P., Marlet, R.: Global fusion of relative motions for robust, accurate and scalable structure from motion. In: 2013 IEEE International Conference on Computer Vision (2013)Google Scholar
  26. 26.

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.University of IllinoisUrbana-ChampaignUSA
  2. 2.Reconstruct Inc.ChampaignUSA

Personalised recommendations