Advertisement

Handcrafted Outlier Detection Revisited

Conference paper
  • 769 Downloads
Part of the Lecture Notes in Computer Science book series (LNCS, volume 12364)

Abstract

Local feature matching is a critical part of many computer vision pipelines, including among others Structure-from-Motion, SLAM, and Visual Localization. However, due to limitations in the descriptors, raw matches are often contaminated by a majority of outliers. As a result, outlier detection is a fundamental problem in computer vision and a wide range of approaches, from simple checks based on descriptor similarity to geometric verification, have been proposed over the last decades. In recent years, deep learning-based approaches to outlier detection have become popular. Unfortunately, the corresponding works rarely compare with strong classical baselines. In this paper we revisit handcrafted approaches to outlier filtering. Based on best practices, we propose a hierarchical pipeline for effective outlier detection as well as integrate novel ideas which in sum lead to an efficient and competitive approach to outlier rejection. We show that our approach, although not relying on learning, is more than competitive to both recent learned works as well as handcrafted approaches, both in terms of efficiency and effectiveness. The code is available at https://github.com/cavalli1234/AdaLAM.

Keywords

Low-level vision Matching Spatial matching Spatial consistency Spatial verification 

Notes

Acknowledgements

This work was supported by a Google Focused Research Award, by the Swedish Foundation for Strategic Research (Semantic Mapping and Visual Navigation for Smart Robots), the Chalmers AI Research Centre (CHAIR) (VisLocLearn) and Innosuisse funding (Grant No. 34475.1 IP-ICT). Viktor Larsson was supported by an ETH Zurich Postdoctoral Fellowship.

Supplementary material

504475_1_En_45_MOESM1_ESM.pdf (18.2 mb)
Supplementary material 1 (pdf 18658 KB)

References

  1. 1.
    Albarelli, A., Rodola, E., Torsello, A.: Robust game-theoretic inlier selection for bundle adjustment. In: International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT2010) (2010)Google Scholar
  2. 2.
    Avrithis, Y., Tolias, G.: Hough pyramid matching: speeded-up geometry re-ranking for large scale image retrieval. Int. J. Comput. Vis. (IJCV) 107(1), 1–19 (2014).  https://doi.org/10.1007/s11263-013-0659-3CrossRefGoogle Scholar
  3. 3.
    Bailey, T., Durrant-Whyte, H.: Simultaneous localization and mapping (slam): part ii. IEEE Rob. Autom. Mag. 13(3), 108–117 (2006)CrossRefGoogle Scholar
  4. 4.
    Barath, D., Matas, J.: Graph-cut RANSAC. In: Computer Vision and Pattern Recognition (CVPR) (2018)Google Scholar
  5. 5.
    Barath, D., Matas, J., Noskova, J.: MAGSAC: marginalizing sample consensus. In: Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar
  6. 6.
    Bian, J., Lin, W.Y., Matsushita, Y., Yeung, S.K., Nguyen, T.D., Cheng, M.M.: GMS: grid-based motion statistics for fast, ultra-robust feature correspondence. In: Computer Vision and Pattern Recognition (CVPR) (2017)Google Scholar
  7. 7.
    Brachmann, E., Rother, C.: Neural-guided RANSAC: learning where to sample model hypotheses. In: International Conference on Computer Vision (ICCV) (2019)Google Scholar
  8. 8.
    Cech, J., Matas, J., Perdoch, M.: Efficient sequential correspondence selection by cosegmentation. Trans. Pattern Anal. Mach. Intell. (PAMI) 32(9), 1568–1581 (2010)CrossRefGoogle Scholar
  9. 9.
    Chum, O., Matas, J.: Matching with PROSAC-progressive sample consensus. In: Computer Vision and Pattern Recognition (CVPR) (2005)Google Scholar
  10. 10.
    Chum, O., Matas, J.: Optimal randomized RANSAC. Trans. Pattern Anal. Mach. Intell. (PAMI) 30(8), 1472–1482 (2008)CrossRefGoogle Scholar
  11. 11.
    Chum, O., Matas, J., Kittler, J.: Locally optimized RANSAC. In: Michaelis, B., Krell, G. (eds.) DAGM 2003. LNCS, vol. 2781, pp. 236–243. Springer, Heidelberg (2003).  https://doi.org/10.1007/978-3-540-45243-0_31CrossRefGoogle Scholar
  12. 12.
    Dang, Z., Yi, K.M., Hu, Y., Wang, F., Fua, P., Salzmann, M.: Eigendecomposition-free training of deep networks with zero eigenvalue-based losses. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11209, pp. 792–807. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01228-1_47CrossRefGoogle Scholar
  13. 13.
    Durrant-Whyte, H., Bailey, T.: Simultaneous localization and mapping: part i. IEEE Rob. Autom. Mag. 13(2), 99–110 (2006)CrossRefGoogle Scholar
  14. 14.
    Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Hartley, R.I., Sturm, P.: Triangulation. Comput. Vis. Image Underst. (CVIU) 68(2), 146–157 (1997)CrossRefGoogle Scholar
  16. 16.
    Heinly, J., Schönberger, J.L., Dunn, E., Frahm, J.M.: Reconstructing the world* in six days *(as captured by the yahoo 100 million image dataset). In: Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  17. 17.
    Jegou, H., Douze, M., Schmid, C.: Hamming embedding and weak geometric consistency for large scale image search. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5302, pp. 304–317. Springer, Heidelberg (2008).  https://doi.org/10.1007/978-3-540-88682-2_24CrossRefGoogle Scholar
  18. 18.
    Johns, E., Yang, G.Z.: RANSAC with 2D geometric cliques for image retrieval and place recognition. In: Computer Vision and Pattern Recognition Workshops (CVPRW) (2015)Google Scholar
  19. 19.
    Jung, I.K., Lacroix, S.: A robust interest points matching algorithm. In: International Conference on Computer Vision (ICCV) (2001)Google Scholar
  20. 20.
    Köser, K.: Geometric estimation with local affine frames and free-form surfaces. Ph.D. thesis, University of Kiel (2009). http://d-nb.info/994782322
  21. 21.
    Lebeda, K., Matas, J., Chum, O.: Fixing the locally optimized RANSAC-full experimental evaluation. In: British Machine Vision Conference (BMVC) (2012)Google Scholar
  22. 22.
    Leordeanu, M., Hebert, M.: A spectral technique for correspondence problems using pairwise constraints. In: International Conference on Computer Vision (ICCV) (2005)Google Scholar
  23. 23.
    Li, X., Larson, M., Hanjalic, A.: Pairwise geometric matching for large-scale object retrieval. In: Computer Vision and Pattern Recognition (CVPR) (2015)Google Scholar
  24. 24.
    Li, Y., Snavely, N., Huttenlocher, D.P.: Location recognition using prioritized feature matching. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010. LNCS, vol. 6312, pp. 791–804. Springer, Heidelberg (2010).  https://doi.org/10.1007/978-3-642-15552-9_57CrossRefGoogle Scholar
  25. 25.
    Lin, W.-Y., Liu, S., Jiang, N., Do, M.N., Tan, P., Lu, J.: RepMatch: robust feature matching and pose for reconstructing modern cities. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 562–579. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46448-0_34CrossRefGoogle Scholar
  26. 26.
    Lin, W.Y., et al.: CODE: coherence based decision boundaries for feature correspondence. Trans. Pattern Anal. Mach. Intell. (PAMI) 40(1), 34–47 (2017)CrossRefGoogle Scholar
  27. 27.
    Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. (IJCV) 60(2), 91–110 (2004).  https://doi.org/10.1023/B:VISI.0000029664.99615.94CrossRefGoogle Scholar
  28. 28.
    Ma, J., Zhao, J., Jiang, J., Zhou, H., Guo, X.: Locality preserving matching. Int. J. Comput. Vis. (IJCV) 127(5), 512–531 (2019).  https://doi.org/10.1007/s11263-018-1117-zMathSciNetCrossRefGoogle Scholar
  29. 29.
    Montemerlo, M., Thrun, S., Koller, D., Wegbreit, B., et al.: FastSLAM: a factored solution to the simultaneous localization and mapping problem. In: Conference on Artificial Intelligence (AAAI) (2002)Google Scholar
  30. 30.
    Yi, K.M., Trulls, E., Ono, Y., Lepetit, V., Salzmann, M., Fua, P.: Learning to find good correspondences. In: Computer Vision and Pattern Recognition (CVPR) (2018)Google Scholar
  31. 31.
    Ni, K., Jin, H., Dellaert, F.: GroupSAC: efficient consensus in the presence of groupings. In: International Conference on Computer Vision (ICCV) (2009)Google Scholar
  32. 32.
    Raguram, R., Chum, O., Pollefeys, M., Matas, J., Frahm, J.M.: USAC: a universal framework for random sample consensus. Trans. Pattern Anal. Mach. Intell. (PAMI) 35(8), 2022–2038 (2012)CrossRefGoogle Scholar
  33. 33.
    Ranftl, R., Koltun, V.: Deep fundamental matrix estimation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11205, pp. 292–309. Springer, Cham (2018).  https://doi.org/10.1007/978-3-030-01246-5_18CrossRefGoogle Scholar
  34. 34.
    Rocco, I., Cimpoi, M., Arandjelović, R., Torii, A., Pajdla, T., Sivic, J.: Neighbourhood consensus networks. In: Neural Information Processing Systems (NeurIPS) (2018)Google Scholar
  35. 35.
    Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: International Conference on Computer Vision (ICCV) (2011)Google Scholar
  36. 36.
    Sarlin, P.E., Cadena, C., Siegwart, R., Dymczyk, M.: From coarse to fine: robust hierarchical localization at large scale. In: Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar
  37. 37.
    Sarlin, P.E., DeTone, D., Malisiewicz, T., Rabinovich, A.: SuperGlue: learning feature matching with graph neural networks. In: Computer Vision and Pattern Recognition (CVPR), pp. 4938–4947 (2020)Google Scholar
  38. 38.
    Sattler, T., Leibe, B., Kobbelt, L.: SCRAMSAC: improving RANSAC’s efficiency with a spatial consistency filter. In: International Conference on Computer Vision (ICCV) (2009)Google Scholar
  39. 39.
    Sattler, T., et al.: Benchmarking 6DOF outdoor visual localization in changing conditions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8601–8610 (2018)Google Scholar
  40. 40.
    Schönberger, J.L., Price, T., Sattler, T., Frahm, J.-M., Pollefeys, M.: A vote-and-verify strategy for fast spatial verification in image retrieval. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10111, pp. 321–337. Springer, Cham (2017).  https://doi.org/10.1007/978-3-319-54181-5_21CrossRefGoogle Scholar
  41. 41.
    Schönberger, J.L., Frahm, J.M.: Structure-from-motion revisited. In: Computer Vision and Pattern Recognition (CVPR) (2016)Google Scholar
  42. 42.
    Schönberger, J.L., Zheng, E., Frahm, J.-M., Pollefeys, M.: Pixelwise view selection for unstructured multi-view stereo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 501–518. Springer, Cham (2016).  https://doi.org/10.1007/978-3-319-46487-9_31CrossRefGoogle Scholar
  43. 43.
    Sivic, J., Zisserman, A.: Efficient visual search of videos cast as text retrieval. Trans. Pattern Anal. Mach. Intell. (PAMI) 31(4), 591–606 (2008)CrossRefGoogle Scholar
  44. 44.
    Strecha, C., Von Hansen, W., Van Gool, L., Fua, P., Thoennessen, U.: On benchmarking camera calibration and multi-view stereo for high resolution imagery. In: Computer Vision and Pattern Recognition (CVPR) (2008)Google Scholar
  45. 45.
    Sturm, J., Engelhard, N., Endres, F., Burgard, W., Cremers, D.: A benchmark for the evaluation of RGB-D SLAM systems. In: International Conference on Intelligent Robots and Systems (IROS) (2012)Google Scholar
  46. 46.
    Thomee, B., Shamma, D.A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., Li, L.J.: YFCC100M: the new data in multimedia research. Commun. ACM 59(2), 64–73 (2016)CrossRefGoogle Scholar
  47. 47.
    Torr, P.H., Nasuto, S.J., Bishop, J.M.: NAPSAC: high noise, high dimensional robust estimation-it’s in the bag. In: British Machine Vision Conference (BMVC) (2002)Google Scholar
  48. 48.
    Torr, P.H., Zisserman, A.: MLESAC: a new robust estimator with application to estimating image geometry. Comput. Vis. Image Underst. (CVIU) 78(1), 138–156 (2000)CrossRefGoogle Scholar
  49. 49.
    Ullman, S.: The interpretation of structure from motion. Proc. R. Soc. Lond. B Biol. Sci. 203(1153), 405–426 (1979)CrossRefGoogle Scholar
  50. 50.
    Wu, C.: SiftGPU: a GPU implementation of scale invariant feature transform (SIFT) (2011). http://cs.unc.edu/~ccwu/siftgpu
  51. 51.
    Wu, C., Agarwal, S., Curless, B., Seitz, S.M.: Multicore bundle adjustment. In: Computer Vision and Pattern Recognition (CVPR) (2011)Google Scholar
  52. 52.
    Wu, C., et al.: VisualSFM: a visual structure from motion system (2011)Google Scholar
  53. 53.
    Wu, X., Kashino, K.: Adaptive dither voting for robust spatial verification. In: International Conference on Computer Vision (ICCV) (2015)Google Scholar
  54. 54.
    Wu, X., Kashino, K.: Robust spatial matching as ensemble of weak geometric relations. In: British Machine Vision Conference (BMVC) (2015)Google Scholar
  55. 55.
    Wu, Z., Ke, Q., Isard, M., Sun, J.: Bundling features for large scale partial-duplicate web image search. In: Computer Vision and Pattern Recognition (CVPR) (2009)Google Scholar
  56. 56.
    Xiao, J., Owens, A., Torralba, A.: SUN3D: a database of big spaces reconstructed using SFM and object labels. In: International Conference on Computer Vision (ICCV) (2013)Google Scholar
  57. 57.
    Zhang, J., et al.: Learning two-view correspondences and geometry using order-aware network (2019)Google Scholar
  58. 58.
    Zhang, Z., Deriche, R., Faugeras, O., Luong, Q.T.: A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artif. Intell. 78(1–2), 87–119 (1995)CrossRefGoogle Scholar
  59. 59.
    Zhao, C., Cao, Z., Li, C., Li, X., Yang, J.: NM-Net: mining reliable neighbors for robust feature correspondences. In: Computer Vision and Pattern Recognition (CVPR) (2019)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.Department of Computer ScienceETH ZurichZürichSwitzerland
  2. 2.Chalmers University of TechnologyGothenburgSweden
  3. 3.Microsoft Mixed Reality & AI Zurich LabZürichSwitzerland

Personalised recommendations